Pixel-wise point spread function engineering systems and methods

ABSTRACT

Systems, devices, and methods for producing an optimized phase mask for use in a single-molecule orientation localization microscopy (SMOLM) imaging system are disclosed.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority from U.S. Provisional Application Ser.No. 63/228,868 filed on Aug. 3, 2021, the content of which isincorporated herein by reference in its entirety.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

This invention was made with government support under 1653777 awarded bythe National Science Foundation and GM124858 awarded by the NationalInstitutes of Health. The government has certain rights in theinvention.

MATERIAL INCORPORATED BY REFERENCE

Not applicable.

FIELD OF THE DISCLOSURE

The present disclosure generally relates to single-molecule orientationlocalization microscopy (SMOLM) systems and methods and in particular tomethods for producing optical masks for use in SMOLM systems.

BACKGROUND OF THE DISCLOSURE

In soft matter, thermal energy causes molecules to continuouslytranslate and rotate, even in crowded environments, impacting thespatial organization and function of most molecular assemblies, such aslipid membranes. At the bulk level, these dynamics are typicallymeasured using absorption, fluorescence, nuclear magnetic resonance, orRaman spectroscopies. Directly measuring the orientation and spatialorganization of large collections of single molecules remains elusive,particularly with high sampling densities (>900 molecules/μm²) andnanoscale resolution.

Tracking a molecule's 3D position and orientation (and associatedtranslational and rotational motions) within soft matter is critical forunderstanding the intrinsically heterogeneous and complex interactionsof its various components across length scales, including associationsof surrounding molecules, functional groups, ions, and charges. Inliving cells, the local organization of and interfaces between manybiomolecular assemblies, such as lipid membranes, chromosomes, andcytoskeletal proteins, ensure the proper functioning of all cellularcompartments. The orientation and organization of molecules alsosignificantly impact the nanoscale morphology of supramolecularstructures, the physical and mechanical properties of polymers, and thecarrier mobility in organic solar cells and organic light-emittingdiodes.

Molecular orientations are commonly inferred from measuring an orderparameter determined via X-ray diffraction, infrared spectroscopy, NMR,Raman spectroscopy, sum frequency generation spectroscopy, andfluorescence microscopy. However, the order parameter is an ensembleaverage taken over many molecules and cannot unambiguously determine the3D orientation of a single molecule (SM). Spectrally-resolved SMlocalization microscopy (SMLM) has been developed to map the localpolarity or hydrophobicity of protein aggregates and subcellularstructures, and fluorescence lifetime imaging has been applied torecognize sub-resolution lipid domains in the plasma membrane. However,these approaches require specific environment-sensitive fluorescentprobes (e.g., Nile red, Laurdan, and 3-hydroxyflavone derivatives) whoseexcited electronic states are sensitive to the local environment,resulting in detectable changes in fluorescence spectra (intensities) orlifetimes. Alternatively, the orientation and motion of any fluorescentprobe are directly influenced by its local environment, regardless ofits solvatochromicity or lifetime. Therefore, imaging the 3D orientationand wobble of SMs (i.e. “orientation spectra”) offers an alternative andwidely applicable strategy for sensing molecular interactions within asample of interest using any SMLM-compatible fluorescent dyes.Orientation spectra, which are characteristics of the molecules, may beinferred from angular emission spectra and polarization spectra, whichare characteristics of the detected photons. Nanoscale imaging of SMorientation spectra may provide direct insight into the spatialorganization of molecular assemblies, macromolecules, and subcellularstructures, which is helpful for constructing mechanistic models ofbiological systems.

SUMMARY OF THE DISCLOSURE

In one aspect, a computer-implemented method for producing an opticalmask for an SMOLM imaging system is disclosed that includes providing,to a computing device, a baseline optical mask comprising a plurality ofmask pixels distributed at a plurality of mask pixel positions within amask plane, each mask pixel comprising at least one optical modulationelement configured to modulate at least one optical parameter of aphoton produced by an emitter propagating therethrough. The methodfurther includes providing, to the computing device, a plurality ofemitter images indicative of dipole spread functions captured using theSMOLM imaging system provided with the baseline optical mask, each imagecomprising a plurality of image pixels indicative of a dipole spreadfunction, each image pixel comprising a pixel position and a pixelintensity indicative of a number of photons detected at the pixelposition, wherein each emitter image is obtained for a reference emitterpositioned at a reference lateral position and at one sample orientationwithin an orientation space. The method further includes determining,using the computing device, a loss function comprising a matrixquantifying variances in precision of emitter orientations estimatedfrom the dipole spread functions from the plurality of images. Themethod further includes iteratively modifying, using the computingdevice, at least one optical parameter of at least one mask pixel tominimize the loss function to produce the optical mask. In some aspects,the at least one optical parameter modulated by each mask pixelcomprises a phase, a polarization, a birefringence, and any combinationthereof. In some aspects, the optical parameter modulated by each maskpixel is the phase. In some aspects, the plurality of emitter images areobtained by imaging an emitter at a plurality of orientations within theorientation space using the SMOLM imaging system provided with abaseline optical mask or a modified optical mask; or simulating eachemitter image using a computational model of a SMOLM imaging systemprovided with a baseline optical mask or a modified optical mask. Insome aspects, each emitter image is simulated using a dipole-dipolemodel. In some aspects, the matrix quantifying variances in precision ofemitter orientations estimated from the dipole spread functions from theplurality of images comprises a Cramér-Rao bound matrix K, wherein theCramér-Rao bound matrix quantifies a lower bound on a variance ofestimated emitter orientations. In some aspects, the loss function l isgiven by:

${l = {\min\limits_{P}{\sum_{m \in {\mathbb{C}}}\sqrt{K\left( {P,m} \right)}}}},$

where m is a second-moment vector m of a dipole spread function obtainedusing the optical mask P, and C is a uniformly sampled emitterorientation space. In some aspects, the method further includesproducing additional optical masks for images of an emitter positionedat different axial positions within the SMOLM imaging system, for imagesof an emitter comprising background photons, or for images of emitterspositioned out of the focal plane of the SMOLM imaging system. In someaspects, the loss function is quantified and minimized by a divergencestatistical model. In some aspects, each emitter image of the pluralityof images is indicative of at least two dipole spread functionscorresponding to at least two emitters within each image.

In one aspect, a computer-implemented method for producing a phase maskfor an SMOLM imaging system is disclosed that includes providing, to acomputing device, a baseline optical mask comprising a plurality of maskpixels distributed at a plurality of mask pixel positions within a maskplane, each mask pixel comprising at least one optical modulationelement configured to modulate a phase of a photon produced by anemitter propagating therethrough. The method further includes providing,to the computing device, a plurality of emitter images indicative ofdipole spread functions captured using the SMOLM imaging system providedwith the baseline optical mask, each image comprising a plurality ofimage pixels indicative of a dipole spread function, each image pixelcomprising a pixel position and a pixel intensity indicative of a numberof photons detected at the pixel position, wherein each emitter image isobtained for a reference emitter positioned at a reference lateralposition and at one sample orientation within an orientation space. Themethod further includes determining, using the computing device, a lossfunction comprising a matrix quantifying variances in precision ofemitter orientations estimated from the dipole spread functions from theplurality of images. The method further includes iteratively modifying,using the computing device the phase of at least one mask pixel tominimize the loss function to produce the optical mask. In some aspects,the plurality of emitter images are obtained by imaging an emitter at aplurality of orientations within the orientation space using the SMOLMimaging system provided with a baseline phase mask or a modified phasemask; or simulating each emitter image using a computational model of aSMOLM imaging system provided with a baseline phase mask or a modifiedphase mask. In some aspects, each emitter image is simulated using adipole-dipole model. In some aspects, the matrix quantifying variancesin precision of emitter orientations estimated from the dipole spreadfunctions from the plurality of images comprises a Cramér-Rao boundmatrix K, wherein the Cramér-Rao bound matrix quantifies a lower boundon a variance of estimated emitter orientations. In some aspects, theloss function l is given by:

${l = {\min\limits_{P}{\sum_{m \in {\mathbb{C}}}\sqrt{K\left( {P,m} \right)}}}},$

where m is a second-moment vector m of a dipole spread function obtainedusing the optical mask P, and C is a uniformly sampled emitterorientation space. In some aspects, the method further includesproducing additional phase masks for images of an emitter positioned atdifferent axial positions within the SMOLM imaging system, for images ofan emitter comprising background photons, or for images of emitterspositioned out of the focal plane of the SMOLM imaging system. In someaspects, the loss function is quantified and minimized by a divergencestatistical model. In some aspects, each emitter image of the pluralityof images is indicative of at least two dipole spread functionscorresponding to at least two emitters within each image.

Other objects and features will be in part apparent and in part pointedout hereinafter.

DESCRIPTION OF THE DRAWINGS

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee.

Those of skill in the art will understand that the drawings, describedbelow, are for illustrative purposes only. The drawings are not intendedto limit the scope of the present teachings in any way.

FIG. 1 is a schematic drawing illustrating a dipole emitterrepresentation of a light-emitting molecule.

FIG. 2 is a schematic drawing illustrating a single molecule microscopysystem with a phase mask inserted at the back focal plane (BFP).

FIG. 3 contains a series of images showing basis images at the backfocal plane (upper) and the image plane (lower).

FIG. 4 is a schematic representation of the method of obtaining a 3Dorientation vector from an intensity distribution captured by a cameraof a single molecule microscope system.

FIG. 5 is a schematic representation of a method for optimizing a phasemask for a single molecule microscopy system.

FIG. 6 is a block diagram schematically illustrating a system inaccordance with one aspect of the disclosure.

FIG. 7 is a block diagram schematically illustrating a computing devicein accordance with one aspect of the disclosure.

FIG. 8 is a block diagram schematically illustrating a remote or usercomputing device in accordance with one aspect of the disclosure.

FIG. 9 is a block diagram schematically illustrating a server system inaccordance with one aspect of the disclosure.

FIG. 10 is a schematic illustration of traditional imaging of afluorescent sample that takes advantage of the orientation and rotationof a molecule. An image of a fluorescent emitter, e.g., a molecule ornanoparticle, depends on its orientation and axial location (h). Theimage also contains information on how much a molecule rotates during acamera frame (called its wobbling). Individual fluorescent molecules canbe modeled as dipoles and it can be assumed a molecule rotates (wobbles)within a symmetric cone during one exposure time. Then θ, ϕ can be usedto describe the center orientation of the cone and use solid angle Ω[sr] to describe the wobbling unit area on the unit sphere (Ω=0 meansfixed dipole emitter and Ω=2π means a freely rotating, isotropicemitter). Each emitter also has its distinct 3D location (x, y, h).Completed measurements of the six-dimensional information (θ, ψ, Ω, x,y, h) can be a challenge since the information of orientation andposition are coupled in the captured image. The traditional imagingsystem (that creates the standard dipole spread function (DSF)) performspoorly to convey the orientation (θ, ϕ, Ω) and the axial location (h) tothe image captured by a camera.

FIG. 11 is a schematic of the imaging scheme of the current disclosurewherein the emission light is manipulated by inserting a pixel-wiseoptimized phase mask P at the back focal plane (BFP). The polarizingbeam splitter (PBS) positioned after the BFP splits the light into twopolarization channels. The light is then captured by two cameras or twoseparate regions of a camera. The Cramér-Rao bound (CRB) matrix R_(M)for orientational second moments indicates the orientational estimationprecision of a technique. A gradient descent algorithm at TensorFlowplatform is built to optimize the N×N pixels of the phase mask Psimultaneously by minimizing the sum of the square root of thedeterminant of R_(M) over a uniformly sampled orientation space. We termthe optimized technique pixOL.

FIG. 12 is a set of images showing how pixOL encodes the orientation andthe axial location information to the dipole spread function wherein theorientation and the 3D location can be measured unambiguously. Theimages show six distinct emitters with difference orientation at twoaxial positions (h=0 and 300 nm). Emitter 1: an isotropic emitter withΩ=2π; emitter 2: θ=30°, φ=45°, Ω=0; emitter 3: θ=90°, φ=45°, Ω=0;emitter 4: θ=0°, Ω=0; emitter 5: θ=30°, φ=0°, Ω=0; emitter 6: θ=90°,φ=0°, 2=0.

FIG. 13 is a set of graphs comparing DSF methods (tri-spot, CHIDO,defocused polarization, optimized pixOL) calculating the best achievableestimation precision calculated from Cramér-Rao bound theory. pixOLprovides smaller uncertainty for θ, ϕ, Ω estimation compared to existingDSFs and also has a good lateral and axial precision. The root-meansquare angular error (RMSAE) σ_(k) is a combined metric from thestandard derivation (std) σ_(θ) of θ estimation and the std σ_(ϕ) of ϕestimation (σ_(k)=√{square root over (sin² θ σ_(ϕ) ²+σ_(θ) ²))}. The stdσ_(r) of lateral precision is a combined metric from the std σ_(x) of xaxis and the std σ_(y) of y axis (σ_(r)=√{square root over (σ_(x)²+σ_(y) ²)}).

FIG. 14 is a set of images comparing pixOL to tri-spot in Poisson noiseand noiseless conditions. The small footprint of pixOL provides ithigher detectability for a localization software than multispot DSFs(e.g., tri-spot). The image also has less overlapped dipole spreadfunctions when the emitters' density is high.

FIG. 15 is a schematic of the pixOL optical set-up.

FIG. 16A is a graph of mean orientation direction accuracy between DSFs(trispot, defocused polarization, CHIDO, pixOL) using theoreticalmodels.

FIG. 16B is a graph of wobbling angle accuracy between DSFs (trispot,defocused polarization, CHIDO, pixOL) using theoretical models.

FIG. 16C is a graph of lateral location accuracy between DSFs (trispot,defocused polarization, CHIDO, pixOL) using theoretical models.

FIG. 16D is a graph of axial location accuracy between DSFs (trispot,defocused polarization, CHIDO, pixOL) using theoretical models.

FIG. 17 is a schematic of a fluorescent contrast agent used in pixOLexperiments composed of a 2 μm diameter silica bead core with a lipidbilayer coating containing Nile red and cholesterol.

FIG. 18 is a set of x-polarized (left) and y-polarized (right) images ofthe contrast agent described in FIG. 17 before (top) and after (bottom)pixOL reconstruction.

FIG. 19 is a set of pixOL images through an individual silica beaddescribed in FIG. 17 , wherein the Nile red is oriented perpendicular tothe spherical surface.

FIG. 20 is a schematic of a fluorescent contrast agent used in pixOLexperiments composed of a 2 μm diameter silica bead core with a lipidbilayer coating containing Nile red.

FIG. 21 is a set of pixOL images through an individual silica beaddescribed in FIG. 20 , wherein the Nile red is randomly oriented on thespherical surface.

FIG. 22A is a reconstructed pixOL image of the bead described in FIG. 20(no cholesterol) showing randomly oriented (θ) Nile red on the surface.

FIG. 22B is a reconstructed pixOL image of the bead described in FIG. 17(with cholesterol) showing Nile red oriented perpendicular (θ) on thesurface.

FIG. 23 is a pixOL image (left) and a comparison of the accuracy of thepixOL image compared to a theoretical pixOL model (middle and right).The experimental precision (FWHM=137 nm) for measuring lateral directionis very close to the theoretical precision (FWHM=82 nm) over an axialrange of 1200 nm.

FIG. 24 is a summary schematic of the pixOL workflow.

FIG. 25A is a set of two raw pixOL images of DPCC coated silica beads(left=red scale, right=blue scale) using 30 orientation measurements.Colorbar: photons. Scale bar: 1 μm.

FIG. 25B is a set of two raw pixOL images of DPCC coated silica beadscontaining 40% cholesterol (left=red scale, right=blue scale) using 30orientation measurements. Colorbar: photons. Scale bar: 1 μm. (c)

FIG. 25C is a set of histograms of measured 3D orientations (polar angleθ, wobbling solid angle Ω) of nile red (NR) contrast agent. Blue: DPPC;orange: DPPC with 40% cholesterol.

FIG. 25D is a scatter plot of the measured θ and Ω of NR in DPPC (blue)and DPPC with cholesterol (orange). Purple cross: median orientations ofNR in DPPC; yellow cross: median orientations of NR in DPPC with 40%cholesterol.

FIG. 26A is a pixOL phase mask imaging system schematic. A microscopeobjective is focused at a nominal focal plane (NFP) (dotted black line)within water at a distance −z above (+z below) the coverslip (z=0). Theobjective collects fluorescence photons from emitters at variouslocations (x,y,h). A polarization-sensitive 4f system, comprising 3lenses and a polarizing beam splitter (PBS), is added after themicroscope's intermediate image plane (IIP) to enable the placement ofthe pixOL optimized phase mask at the back focal plane (BFP). (MDL: movedashed line to be exactly halfway between two 4f lenses.) Two cameras(or two regions of a single cam-era) capture x-polarized (red) andy-polarized (cyan) light.

FIG. 26B is a schematic of parameterizing the orientation of an emitterusing polar angle θε [0°, 90°], azimuthal angle ϕϵ (−180°, 180°], andwobbling solid angle Ωϵ [0, 2π] sr.

FIG. 26C is an image of an optimized pixOL phase mask. Colormap: phase(rad). Scalebar: 500 μm.

FIG. 26D is a set of simulated images of emitters located at h=800 nm,h=400 nm, and h=0 nm with orientations (θ, ϕ, Ω) shown in FIG. 26B(emitter 1: (−, −, 2π), emitter 2: (0°, 0°, 0), emitter 3: (45°, 0°, 0),emitter 4: (90°, 0°, 0)), captured in the two polarization channelsshown in (a) with the objective lens focused at z=−580 nm. Theintensities of each red-blue image pair are normalized. Scale bar: 500nm.

FIG. 27A is a graph of the mean of the mean square angular error (MSAE)σ_(A) best-possible measurement precision of pixOL DSF calculated fromthe Cramér-Rao bound (CRB) matrix K for emitters within water (RI=1.33)with 1000 signal photons and 2 background photons per pixel detectedusing various engineered DSFs. Red: pixOL, yellow: defocused polarizedDSF focused at 200 nm below the coverslip, grey: CHIDO, purple:tri-spot. The MSAE is a combined metric for the standard deviation σ_(θ)of θ and the standard deviation σ_(ϕ), of ϕ.

FIG. 27B is a graph of the average precision σ_(Ω) best-possiblemeasurement precision of pixOL DSF calculated from the Cramér-Rao bound(CRB) matrix K for emitters within water (RI=1.33) with 1000 signalphotons and 2 background photons per pixel detected using variousengineered DSFs. Red: pixOL, yellow: defocused polarized DSF focused at200 nm below the coverslip, grey: CHIDO, purple: tri-spot. Averageprecision σ_(Ω) for measuring the wobbling solid angle Ω.

FIG. 27C is a graph of the average localization of lateral positionσ_(t) best-possible measurement precision of pixOL DSF calculated fromthe Cramér-Rao bound (CRB) matrix K for emitters within water (RI=1.33)with 1000 signal photons and 2 background photons per pixel detectedusing various engineered DSFs. Red: pixOL, yellow: defocused polarizedDSF focused at 200 nm below the coverslip, grey: CHIDO, purple:tri-spot.

FIG. 27D is a graph of the average localization σ_(h) best-possiblemeasurement precision of pixOL DSF calculated from the Cramér-Rao bound(CRB) matrix K for emitters within water (RI=1.33) with 1000 signalphotons and 2 background photons per pixel detected using variousengineered DSFs. Red: pixOL, yellow: defocused polarized DSF focused at200 nm below the coverslip, grey: CHIDO, purple: tri-spot. Height h isabove the interface for an isotropic emitter

FIG. 27E is a plot of the location and orientation estimation of afluorescent bead. The sample is scanned axially from z=−790 nm to z=610nm with a step size of 50 nm (11 camera frames per step). Red dot:estimated focal plane location z in each frame (MDL: change x axis labelto “camera frame”); green cross: expected stage position. Inset (i):experimental axial localization precision σ_(z) at each scanning plane.The mean precision over all z-planes is σ_(z)=2.56 nm. (ii) Experimentalwobble angle precision σ_(Ω) at each scanning plane. The average wobbleangle measurement precision σ_(θ)=0.08 over all scanning planes.

FIG. 28A is a schematic of an experiment measuring the 3D orientationand 3D location of Nile reds on supported lipid bilayers (SLBs). Twotypes of SLBs, namely DPPC with 40% cholesterol (chol) and DPPC, to 2 μmsilica beads are used. The angle θ_(⊥) is defined as the angle betweenthe estimated orientation of Nile red (NR) and the vector μ_(z) _(⊥)perpendicular to the spherical surface. r is used to present thedistance from the estimated location to the vertical diameter of thesphere.

FIG. 28B is an image of the x-h view of the estimated θ for the bottomhalf of bead for a bead coated with DPPC and cholesterol.

FIG. 28C is an image of the x-y view of the estimated θ for the bottomhalf of bead for a bead coated with DPPC and cholesterol.

FIG. 28D is an image of the x-y view of the estimated θ for the bottomhalf of bead for a bead coated with DPPC only.

FIG. 28E is a set of x-y view images of all individual orientationmeasurements in different z-slices along and associated histograms of abead coated with DPPC and cholesterol. The lines are oriented accordingto the 3D mean orientation angle (θ, ϕ) and colors are coded accordingto ϕ angle.

FIG. 28F is a set of histograms (f) Histograms of the measured r for SMsin each z slices of the images in FIG. 28E and are compared to thetheoretical distribution (yellow lines) calculated using the sphericalmodel and the lateral precision of pixOL_(cjg).

FIG. 28G is a sum of images of estimations located within a 50 nm×50nm×100 nm box at three axial locations shown in FIG. 28E. ((1): h=150nm; (2): h=550 nm; (3): h=950 nm). The centers of boxes are located atthe spherical surface and at a direction with ϕ of −135°. Images at thebottom column are simulated images for emitters with perpendicularorientation to the spherical surface.

FIG. 28H is a graph of the full width half maximums (FWHMs) of rdistribution in FIG. 28F at each z slices for the DPPC with cholesterolbead (blue) and the DPPC bead (yellow) and are compared to thetheoretical FWHM (orange).

FIG. 28I is a plot of the orientation (θ_(⊥)) and wobble (Ω)measurements for DPPC with cholesterol (yellow) and DPPC. Crossesindicates the medians for DPPC with cholesterol (blue) and DPPC(Orange). Scale bars: 400 nm.

FIG. 29 is a table of the measurement precision of pixOL compared toother techniques: CHIDO, double helix (DH), and unpolarized vortex. Thebest-possible measurement precisions are calculated using the Cramér-Raobound (CRB) for mean orientation angle [θ, ϕ] (MASD, σ_(δ)), wobbleangle α_(Ω), lateral position σ_(L), and axial position σ_(h). MASDquantifies the combined precision of measuring θ and ϕ as the half-angleof a cone representing orientation uncertainty (Eqn. S23). Theorientation precisions are calculated for fixed emitters (Ω=0 sr) andaveraged over the whole orientation sphere. The localization precisionsare calculated for isotropic emitters (Ω=2π sr). Emitters are locatedwithin water (1.33 refractive index) with a 2500:3 signal to backgroundratio (SBR, signal photons:background photons/pixel)) and a 2500:10 SBR.

FIG. 30 is an imaging system schematic. Fluorescence from singlemolecules is collected by the objective. A polarization-sensitive 4fsystem, comprising 3 lenses (lenses 1-3) and a polarizing beam splitter(PBS), is added after the microscope's intermediate image plane (IIP).The PBS splits the light into x-polarized (red) and y-polarized (cyan)fluorescence. A pyramid mirror is used to reflect light from twochannels onto the spatial light modulator (SLM) placed at the conjugateback focal plane (BFP) of the microscope (insets i and ii). The pixOLphase mask is loaded onto the SLM to modulate the phase of both channelssimultaneously. Lenses 2 and 3 focus the x- and y-polarized fluorescenceonto two non-overlapping regions of a single CMOS camera (camera 1).Arrows denote the polarization of the light in each channel. The systemcan be modified to image the BFP by using the flipping mirror (FM).Camera 2 is shown to be imaging the y-polarized BFP, but the x-polarizedBFP can be imaged by simply translating the camera appropriately. M1-5,mirrors.

FIG. 31 is a set of basis images B_(BFP) at the back-focal plane (BFP)of an x-y polarized microscope (red: x-polarized, blue: y-polarized,FIG. 26A). The radius of the circular support in the BFP corresponds tothe finite numerical aperture of the microscope's objective lens. Theimage intensities are normalized relative to the brightest basis image(B_(XX)). Colorbars: normalized intensity.

FIG. 32 shows the effect of modulating super-critical fluorescencedifferently from sub-critical fluorescence.

FIG. 32A is a set of panels where (i) Pixel-based optimization foremitters located at the interface between a glass and water (refractiveindex=1.33). The masks are optimized for an objective with a numericalaperture of 1.4, (ii) The super-critical regions of the masks in (i) aresmoothed using phase values from the sub-critical regions. (iii) Meanangular standard deviation σ_(δ) and (iv) mean wobbling angle precisionσ_(Ω) averaged over all θ for emitters with 380 signal photons and 2background photons per pixel.

FIG. 32B is a set of panels where (i) Pixel-based optimization foremitters located at the interface between a glass and air (refractiveindex=1). The masks are optimized for an objective with a numericalaperture of 1.4. (ii) The super-critical regions of the masks in (i) aresmoothed using phase values from the sub-critical regions. (iii) Meanangular standard deviation σ_(δ) and (iv) mean wobbling angle precisionσ_(Ω) averaged over all θ for emitters with 380 signal photons and 2background photons per pixel.

FIG. 32C is a set of panels where (i) Pixel-based optimization foremitters located at the interface between a glass and water (refractiveindex=1.33). The masks are optimized for an objective with a numericalaperture of 1.5. (ii) The super-critical regions of the masks in (i) aresmoothed using phase values from the sub-critical regions. (iii) Meanangular standard deviation σ_(δ) and (iv) mean wobbling angle precisionσ_(Ω) averaged over all θ for emitters with 380 signal photons and 2background photons per pixel.

FIG. 33A is a set of panels where (i) Pixel-based optimization and (ii)Zernike polynomial-based optimization for emitters located at theinterface between a glass coverslip and water (refractive index=1.33).The masks are optimized for an objective with a numerical aperture of1.4. (iii) Zernike polynomial decompositions of (blue) the pixel-wiseoptimized mask in (i) and (orange) the Zernike-optimized mask in (ii). Yaxis: magnitude of the ith Zernike polynomial. (iv) Phase maskreconstructed from the Zernike decomposition (blue curve in (iii)) ofthe pixel-based optimized phase mask in (i) using the first 55 Zernikepolynomials. (iv) Optimization loss curves for the (blue) pixel-wiseoptimized mask in (i) and (orange) the Zernike-optimized mask in (iii).Inset: the loss curves at the last 50 optimization steps. The lossfunction is calculated using

(Eqn. 11) for emitters with a total signal photons of 380 and 2background photons/pixel.

FIG. 33B is a set of panels where (i) Pixel-based optimization and (ii)Zernike polynomial-based optimization for emitters located at theinterface between a glass coverslip and air (refractive index=1). Themasks are optimized for an objective with a numerical aperture of 1.4.(iii) Zernike polynomial decompositions of (blue) the pixel-wiseoptimized mask in (i) and (orange) the Zernike-optimized mask in (ii). Yaxis: magnitude of the ith Zernike polynomial. (iv) Phase maskreconstructed from the Zernike decomposition (blue curve in (iii)) ofthe pixel-based optimized phase mask in (i) using the first 55 Zernikepolynomials. (iv) Optimization loss curves for the (blue) pixel-wiseoptimized mask in (i) and (orange) the Zernike-optimized mask in (iii).Inset: the loss curves at the last 50 optimization steps. The lossfunction is calculated using l (Eqn. 11) for emitters with a totalsignal photons of 380 and 2 background photons/pixel.

FIG. 33C is a set of panels where (i) Pixel-based optimization and (ii)Zernike polynomial-based optimization for emitters located at theinterface between a glass coverslip and water (refractive index=1.33).The masks are optimized for an objective with a numerical aperture of1.5. (iii) Zernike polynomial decompositions of (blue) the pixel-wiseoptimized mask in (i) and (orange) the Zernike-optimized mask in (ii). Yaxis: magnitude of the ith Zernike polynomial. (iv) Phase maskreconstructed from the Zernike decomposition (blue curve in (iii)) ofthe pixel based optimized phase mask in (i) using the first 55 Zernikepolynomials. (iv) Optimization loss curves for the (blue) pixel-wiseoptimized mask in (i) and (orange) the Zernike-optimized mask in (iii).Inset: the loss curves at the last 50 optimization steps. The lossfunction is calculated using

(Eqn. 11) for emitters with a total signal photons of 380 and 2background photons/pixel.

FIG. 34 is a set of image plane basis images B corresponding to thepolarization-sensitive imaging system (red: x-polarized, blue:y-polarized) and pixOL phase mask for an in-focus emitter. The imageintensities are normalized relative to the brightest basis image(B_(XX)). Colorbars: normalized intensity. Scalebar: 500 nm.

FIG. 35 represents optimizing phase masks for various signal tobackground ratios (SBRs, signal photons:background photons/pixel).

FIG. 35A is a Phase mask optimized for emitters with SBR of 380:10,Colorbar: phase (rad).

FIG. 35B is a Phase mask optimized for emitters with SBR of 380:2.Colorbar: phase (rad).

FIG. 35C is a Phase mask optimized for emitters with SBR of 3800:10.Colorbar: phase (rad).

FIG. 35D is a Phase mask optimized for emitters with SBR of 3800:2.Colorbar: phase (rad).

FIG. 35E is a graph of the best-possible measurement precision ofvarious engineered DSFs calculated using the Cramér-Rao bound (CRB) formean angular standard deviation σ_(δ) averaged uniformly over all θ.Grey: mask in FIG. 35A, red: mask in 35B, yellow: mask in FIG. 35C, andpurple: mask in FIG. 35D. Precision is calculated for emitters at awater-glass interface with SBRs of 380:10.

FIG. 35F is a graph of the best-possible measurement precision ofvarious engineered DSFs calculated using the Cramér-Rao bound (CRB) formean angular standard deviation σ_(δ) and averaged uniformly over all θ.Grey: mask in FIG. 35A, red: mask in FIG. 35B, yellow: mask in FIG. 35C,and purple: mask in FIG. 35D. Precision is calculated for emitters at awater-glass interface with SBRs of 380:2.

FIG. 35G is a graph of the best-possible measurement precision ofvarious engineered DSFs calculated using the Cramér-Rao bound (CRB) formean angular standard deviation σ_(δ) averaged uniformly over all θ.Grey: mask in FIG. 35A, red: mask in FIG. 35B, yellow: mask in FIG. 35C,and purple: mask in FIG. 35D. Precision is calculated for emitters at awater-glass interface with SBRs of 3800:10.

FIG. 35H is a graph of the best-possible measurement precision ofvarious engineered DSFs calculated using the Cramér-Rao bound (CRB) formean angular standard deviation σ_(δ) and averaged uniformly over all θ.Grey: mask in FIG. 35A, red: mask in FIG. 35B, yellow: mask in FIG. 35C,and purple: mask in FIG. 35D. Precision is calculated for emitters at awater-glass interface with SBRs of 3800:2.

FIG. 35I is a graph of the best-possible measurement precision ofvarious engineered DSFs calculated using the Cramér-Rao bound (CRB) formean wobble angle precision σ_(Ω) averaged uniformly over all θ. Grey:mask in FIG. 35A, red: mask in FIG. 35B, yellow: mask in FIG. 35C, andpurple: mask in (d). Precision is calculated for emitters at awater-glass interface with SBRs of 380:10.

FIG. 35J is a graph of the best-possible measurement precision ofvarious engineered DSFs calculated using the Cramér-Rao bound (CRB) for(e-h) mean angular standard mean wobble angle precision σ_(Ω) averageduniformly over all θ. Grey: mask in FIG. 35A, red: mask in FIG. 35B,yellow: mask in FIG. 35C, and purple: mask in FIG. 35D. Precision iscalculated for emitters at a water-glass interface with SBRs of 380:2.

FIG. 35K is a graph of the best-possible measurement precision ofvarious engineered DSFs calculated using the Cramér-Rao bound (CRB) formean wobble angle precision σ_(Ω) averaged uniformly over all θ. Grey:mask in FIG. 35A, red: mask in FIG. 35B, yellow: mask in FIG. 35C, andpurple: mask in FIG. 35D. Precision is calculated for emitters at awater-glass interface with SBRs of 3800:10.

FIG. 35L is a graph of the best-possible measurement precision ofvarious engineered DSFs calculated using the Cramér-Rao bound (CRB) formean wobble angle precision σ_(Ω) averaged uniformly over all θ. Grey:mask in FIG. 35A, red: mask in FIG. 35B, yellow: mask in FIG. 35C, andpurple: mask in 35D. Precision is calculated for emitters at awater-glass interface with SBRs of 3800:2.

FIG. 36 is a graph of orientation estimation precision of pixOL phasemasks compared to direct back focal plane (BFP) imaging, which achievesperformance close to the best-case, quantum-limited orientationmeasurement precision. Four pixOL masks (FIG. 35 A-D) are optimized foremitters at various signal to background ratios (SBRs, signalphotons:background photons/pixel): 380:10, 380:2, 380:10, and 3800:2.Orientation precision is calculated using the loss function l (Eqn. 11)for emitters with an SBR of 3800:0.

FIG. 37A is a graph of the estimation precision of pixOL compared totechniques designed solely for 3D orientation measurements. Red: pixOL,blue: x-y polarized standard DSF (pol) defocused at 200 nm above thecoverslip, yellow: tri-spot. Mean angular standard deviation σ_(δ)(MASD) averaged uniformly over all θ. MASD quantifies the combinedprecision of measuring θ and ϕ as the half-angle of a cone representingorientation uncertainty (Eqn. S23). MASD is calculated for in-focus SMswith fixed orientation (Ω=0 sr). Emitters are immersed in water (1.33refractive index) with 2500 signal photons and 3 background photons perpixel.

FIG. 37B is a graph of the estimation precision of pixOL compared totechniques designed solely for 3D orientation measurements. Red: pixOL,blue: x-y polarized standard DSF (pol) defocused at 200 nm above thecoverslip, yellow: tri-spot. Mean wobble angle precision σΩ averageduniformly over all θ.

FIG. 37C is a graph of the estimation precision of pixOL compared totechniques designed solely for 3D orientation measurements. Red: pixOL,blue: x-y polarized standard DSF (pol) defocused at 200 nm above thecoverslip, yellow: tri-spot. Localization precisions σ_(L) lateralposition. Localization precisions are for isotropic emitters (Ω=2π sr).Emitters are immersed in water (1.33 refractive index) with 2500 signalphotons and 3 background photons per pixel.

FIG. 37D is a graph of the estimation precision of pixOL compared totechniques designed solely for 3D orientation measurements. Red: pixOL,blue: x-y polarized standard DSF (pol) defocused at 200 nm above thecoverslip, yellow: tri-spot. Localization precisions σ_(h) for measuringaxial location h above the interface. Localization precisions are forisotropic emitters (Ω=27c sr). Emitters are immersed in water (1.33refractive index) with 2500 signal photons and 3 background photons perpixel.

FIG. 38A is a graph of the orientation estimation precision of pixOL foremitters at various axial locations h compared to techniques designedfor 3D orientation and 3D position measurements. Red: pixOL, grey:CHIDO, yellow: double helix (DH), purple: unpolarized vortex. Meanangular standard deviation σ_(δ) (MASD) averaged uniformly over all θand ϕ. MASD quantifies the combined precision of measuring θ and ϕ asthe half-angle of a cone representing orientation uncertainty (Eqn. S23)

FIG. 38B is a graph of the orientation estimation precision of pixOL foremitters at various axial locations h compared to techniques designedfor 3D orientation and 3D position measurements. Red: pixOL, grey:CHIDO, yellow: double helix (DH), purple: unpolarized vortex. Meanwobble angle precision σ_(Ω) averaged uniformly over all θ and ϕ.Emitters have fixed orientations (Ω=0 sr), and are immersed in water(1.33 refractive index) with 2500 signal photons and 3 backgroundphotons per pixel.

FIG. 39 is a set of images of partial derivatives of the first threebasis images (B_(XX), B_(YY), B_(ZZ)) in FIG. 34 with respect to 3Dposition [x, y, h]. Colorbars: photons/nm. Scalebar: 500 nm.

FIG. 40A is a set of images showing the accuracy of the first-orderapproximation of the pixOL DSF. (i) Images I of the pixOL DSF using afully accurate model for isotropic emitters located at [0, 0, p_(x)].The grid spacing is given by [p_(x)=58.5, p_(y)=58.5, p_(h)=50] nm alongx, y, and h. (ii) Images I_(apx) of the pixOL DSF using a first-orderpolynomial approximation (Eqn. S11). The first-order approximation isevaluated relative to the origin [x₀=0, y₀=0, h₀=0]. (iii) Differencebetween I and I_(apx). Images in each panel are normalized relative tothe accurate pixOL DSF I. Scalebar: 500 nm.

FIG. 40B is a set of images showing accuracy of the first-orderapproximation of the pixOL DSF. (i) Images I of the pixOL DSF using afully accurate model for isotropic emitters located at [0, p_(y), 0].The grid spacing is given by [p_(x)=58.5, p_(y)=58.5, p_(h)=50] nm alongx, y, and h. (ii) Images I_(apx) of the pixOL DSF using a first-orderpolynomial approximation (Eqn. S11). The first-order approximation isevaluated relative to the origin [x₀=0, y₀=0, h₀=0]. (iii) Differencebetween I and I_(apx). Images in each panel are normalized relative tothe accurate pixOL DSF I. Scalebar: 500 nm.

FIG. 40C is a set of images showing accuracy of the first-orderapproximation of the pixOL DSF. (i) Images I of the pixOL DSF using afully accurate model for isotropic emitters located at [0, 0, p_(h)].The grid spacing is given by [p_(x)=58.5, p_(y)=58.5, p_(h)=50] nm alongx, y, and h. (ii) Images I_(apx) of the pixOL DSF using a first-orderpolynomial approximation (Eqn. S11). The first-order approximation isevaluated relative to the origin [x₀=0, y₀=0, h₀=0]. (iii) Differencebetween I and I_(apx). Images in each panel are normalized relative tothe accurate pixOL DSF I. Scalebar: 500 nm.

FIG. 41 is a set of average basis images Ba (red: x-polarized, blue:y-polarized) used for detecting single molecules (Eqn. S14c). Basisimages for an emitter at h=0 nm and h=900 nm are averaged together. Theimage intensities are normalized relative to the brightest basis image(B_(XX)). Colorbars: normalized intensity. Scalebar: 500 nm.

FIG. 42A is a set of 2 polarization-resolved images (red: x and blue: y)of Nile red within spherical lipid bilayers (DPPC with cholesterol). Thelateral position [x, y] of each detected molecule is overlaid as crosses(blue cross in the x-polarized image and red cross in the y-polarizedimage). Scale bar: 500 nm. Color bar: photons/pixel.

FIG. 42B is another set of 2 polarization-resolved images (red: x andblue: y) of Nile red within spherical lipid bilayers (DPPC withcholesterol). The lateral position [x, y] of each detected molecule isoverlaid as crosses (blue cross in the x-polarized image and red crossin the y-polarized image). Scale bar: 500 nm. Color bar: photons/pixel.

FIG. 42C is yet another set of 2 polarization-resolved images (red: xand blue: y) of Nile red within spherical lipid bilayers (DPPC withcholesterol). The lateral position [x, y] of each detected molecule isoverlaid as crosses (blue cross in the x-polarized image and red crossin the y-polarized image). Scale bar: 500 nm. Color bar: photons/pixel.

FIG. 42D is yet another set of 2 polarization-resolved images (red: xand blue: y) of Nile red within spherical lipid bilayers (DPPC withcholesterol). The lateral position [x, y] of each detected molecule isoverlaid as crosses (blue cross in the x-polarized image and red crossin the y-polarized image). Scale bar: 500 nm. Color bar: photons/pixel.

FIG. 43A is a grid representing the coupling between different positionand orientation parameters for pixOL compared to other methods. Averageof the entries in the Fisher information matrix K⁻¹ for estimating 3Dorientation, rotational wobble, and 3D position when using pixOL.Averages are computed over all orientations and for emitters locatedwithin an axial range of h∈[0, 800]. To normalize the matrix, elementK_([i,j]) ⁻¹ in ith row and jth column is divided by K_([i,j]) ^(−1/2)K[j,j]^(−1/2). The emitters are immersed in water (1.33 refractiveindex) and imaged using an objective focused at a normal focal plane of−580 nm, collecting 2500 signal photons and 3 background photons perpixel.

FIG. 43B is a grid representing the coupling between different positionand orientation parameters for pixOL compared to other methods. Averageof the entries in the Fisher information matrix K⁻¹ for estimating 3Dorientation, rotational wobble, and 3D position when using CHIDO.Averages are computed over all orientations and for emitters locatedwithin an axial range of h∈[0, 800]. To normalize the matrix, elementK_([i,j]) ⁻¹ in ith row and jth column is divided by K_([i,j]) ^(−1/2)K[j,j]^(−1/2). The emitters are immersed in water (1.33 refractiveindex) and imaged using an objective focused at a normal focal plane of−580 nm, collecting 2500 signal photons and 3 background photons perpixel.

FIG. 43C is a grid representing the coupling between different positionand orientation parameters for pixOL compared to other methods. Averageof the entries in the Fisher information matrix K⁻¹ for estimating 3Dorientation, rotational wobble, and 3D position when using double helix.Averages are computed over all orientations and for emitters locatedwithin an axial range of h∈[0, 800]. To normalize the matrix, elementK_([i,j]) ⁻¹ in ith row and jth column is divided by K_([i,j]) ^(−1/2)K[j,j]^(−1/2). The emitters are immersed in water (1.33 refractiveindex) and imaged using an objective focused at a normal focal plane of−580 nm, collecting 2500 signal photons and 3 background photons perpixel.

FIG. 43D is a grid representing the coupling between different positionand orientation parameters for pixOL compared to other methods. Averageof the entries in the Fisher information matrix K⁻¹ for estimating 3Dorientation, rotational wobble, and 3D position when using unpolarizedvortex. Averages are computed over all orientations and for emitterslocated within an axial range of h∈[0, 800]. To normalize the matrix,element K_([i,j]) ⁻¹ in ith row and jth column is divided by K_([i,j])^(−1/2) K[j,j]^(−1/2). The emitters are immersed in water (1.33refractive index) and imaged using an objective focused at a normalfocal plane of −580 nm, collecting 2500 signal photons and 3 backgroundphotons per pixel.

FIG. 44A is a graph of the difference ratio Y, which quantifies thedifference in measurement precision accounting for orientation-positioncorrelations vs. ignoring them (Eqn. S26), for orientation precisionσ_(δ), averaged over all polar orientations θ. In-focus SMs with fixedorientation (Ω=0 sr) are used. Emitters are immersed in water (1.33refractive index) with 2500 signal photons and 3 background photons perpixel.

FIG. 44B is a graph of the difference ratio Y for wobble angle precisionon. In-focus SMs with fixed orientation (Ω=0 sr) are used. Emitters areimmersed in water (1.33 refractive index) with 2500 signal photons and 3background photons per pixel.

FIG. 44C is a graph of the difference ratio Y for lateral localizationprecision oi. Isotropic emitters (Ω=2π sr) are used. Emitters areimmersed in water (1.33 refractive index) with 2500 signal photons and 3background photons per pixel.

FIG. 44D is a graph of the difference ratio Y for axial localizationprecision 6 h. Isotropic emitters (Ω=2π sr) are used. Emitters areimmersed in water (1.33 refractive index) with 2500 signal photons and 3background photons per pixel.

FIG. 45A is a set of graphs of the precision and accuracy of estimatingσ_(δ) (deg) for emitters at h=0 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=0 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 45B is a set of graphs of the precision and accuracy of estimatingσ_(Ω) (sr) for emitters at h=0 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=0 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 45C is a set of graphs of the precision and accuracy of estimatingδ_(L) (nm) for emitters at h=0 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=0 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 45D is a set of graphs of the precision and accuracy of estimatingδ_(h) (nm) for emitters at h=0 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=0 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 45E is a set of graphs of the precision and accuracy of estimatingδ−δ₀ (deg) for emitters at h=0 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=0 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 45F is a set of graphs of the precision and accuracy of estimatingΩ−Ω₀ (sr) for emitters at h=0 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=0 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 45G is a set of graphs of the precision and accuracy of estimatingL−L₀ (nm) for emitters at h=0 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=0 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 45H is a set of graphs of the precision and accuracy of estimatingh−h₀ (nm) for emitters at h=0 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=0 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 45J is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=0 sr. 200 independent images weregenerated for emitter within water located at h=0 nm with 2500 signalphotons in total and 3 background photons per pixel detected.

FIG. 45K is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=1 sr. 200 independent images weregenerated for emitter within water located at h=0 nm with 2500 signalphotons in total and 3 background photons per pixel detected.

FIG. 45L is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=2 sr. 200 independent images weregenerated for emitter within water located at h=0 nm with 2500 signalphotons in total and 3 background photons per pixel detected.

FIG. 46A is a set of graphs of the precision and accuracy of estimatingσδ (deg) for emitters at h=400 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=400 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 46B is a set of graphs of the precision and accuracy of estimatingσ_(Ω) (sr) for emitters at h=400 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=400 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 46C is a set of graphs of the precision and accuracy of estimatingσ_(L) (nm) for emitters at h=400 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=400 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 46D is a set of graphs of the precision and accuracy of estimatingδ_(h) (nm) for emitters at h=400 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=400 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 46E is a set of graphs of the precision and accuracy of estimatingδ−δ₀ (deg) for emitters at h=400 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=400 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 46F is a set of graphs of the precision and accuracy of estimatingΩ−Ω₀ (sr) for emitters at h=400 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=400 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 46G is a set of graphs of the precision and accuracy of estimatingL−L₀ (nm) for emitters at h=400 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=400 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 46H is a set of graphs of the precision and accuracy of estimatingh−h₀ (nm) for emitters at h=400 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=400 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 46J is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=0 sr. 200 independent images weregenerated for emitter within water located at h=400 nm with 2500 signalphotons in total and 3 background photons per pixel detected.

FIG. 46K is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=1 sr. 200 independent images weregenerated for emitter within water located at h=400 nm with 2500 signalphotons in total and 3 background photons per pixel detected.

FIG. 46L is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=2 sr. 200 independent images weregenerated for emitter within water located at h=400 nm with 2500 signalphotons in total and 3 background photons per pixel detected.

FIG. 47A is a set of graphs of the precision and accuracy of estimatingσ_(δ) (deg) for emitters at h=700 nm with a nominal focal plane placedat z=−580 nm with low background at various orientations (Ω∈{0, 1, 2}sr, θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independentimages were generated for emitter within water located at h=700 nm with2500 signal photons in total and 3 background photons per pixeldetected.

FIG. 47B is a set of graphs of the precision and accuracy of estimatingσ_(Ω) (sr) for emitters at h=700 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=700 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 47C is a set of graphs of the precision and accuracy of estimatingσ_(L) (nm) for emitters at h=700 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=700 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 47D is a set of graphs of the precision and accuracy of estimatingδ_(h) (nm) for emitters at h=700 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=700 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 47E is a set of graphs of the precision and accuracy of estimatingδ−δ₀ (deg) for emitters at h=700 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=700 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 47F is a set of graphs of the precision and accuracy of estimatingΩ−Ω₀ (sr) for emitters at h=700 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=700 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 47G is a set of graphs of the precision and accuracy of estimatingL−L₀ (nm) for emitters at h=700 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=700 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 47H is a set of graphs of the precision and accuracy of estimatingh−h₀ (nm) for emitters at h=700 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=700 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 47J is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=0 sr. 200 independent images weregenerated for emitter within water located at h=700 nm with 2500 signalphotons in total and 3 background photons per pixel detected.

FIG. 47K is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=1 sr. 200 independent images weregenerated for emitter within water located at h=700 nm with 2500 signalphotons in total and 3 background photons per pixel detected.

FIG. 47L is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=2 sr. 200 independent images weregenerated for emitter within water located at h=700 nm with 2500 signalphotons in total and 3 background photons per pixel detected.

FIG. 48A is a set of graphs of the precision and accuracy of estimatingσ_(δ) (deg) for emitters at h=700 nm with a nominal focal plane placedat z=−580 nm after filtering biased estimates at various orientations(Ω∈{0, 1, 2} sr, θ∈[0°, 90°], ϕ∈[0°, 360°)). Estimates >90 nm away fromthe ground truth 2D position (x0, y0) are removed. At each orientation,200 independent images were generated for emitter within water locatedat h=700 nm with 2500 signal photons in total and 3 background photonsper pixel detected.

FIG. 48B is a set of graphs of the precision and accuracy of estimatingσ_(Ω) (sr) for emitters at h=700 nm with a nominal focal plane placed atz=−580 nm after filtering biased estimates at various orientations(Ω∈{0, 1, 2} sr, θ∈[0°, 90°], ϕ∈[0°, 360°)). Estimates >90 nm away fromthe ground truth 2D position (x0, y0) are removed. At each orientation,200 independent images were generated for emitter within water locatedat h=700 nm with 2500 signal photons in total and 3 background photonsper pixel detected.

FIG. 48C is a set of graphs of the precision and accuracy of estimatingσ_(L) (nm) for emitters at h=700 nm with a nominal focal plane placed atz=−580 nm after filtering biased estimates at various orientations(Ω∈{0, 1, 2} sr, θ∈[0°, 90°], ϕ∈[0°, 360°)). Estimates >90 nm away fromthe ground truth 2D position (x0, y0) are removed. At each orientation,200 independent images were generated for emitter within water locatedat h=700 nm with 2500 signal photons in total and 3 background photonsper pixel detected.

FIG. 48D is a set of graphs of the precision and accuracy of estimatingδ_(h) (nm) for emitters at h=700 nm with a nominal focal plane placed atz=−580 nm after filtering biased estimates at various orientations(Ω∈{0, 1, 2} sr, θ∈[0°, 90°], ϕ∈[0°, 360°)). Estimates >90 nm away fromthe ground truth 2D position (x0, y0) are removed. At each orientation,200 independent images were generated for emitter within water locatedat h=700 nm with 2500 signal photons in total and 3 background photonsper pixel detected.

FIG. 48E is a set of graphs of the precision and accuracy of estimatingδ−δ₀ (deg) for emitters at h=700 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=0 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 48F is a set of graphs of the precision and accuracy of estimatingΩ−Ω₀ (sr) for emitters at h=700 nm with a nominal focal plane placed atz=−580 nm after filtering biased estimates at various orientations(Ω∈{0, 1, 2} sr, θ∈[0°, 90°], ϕ∈[0°, 360°)). Estimates >90 nm away fromthe ground truth 2D position (x0, y0) are removed. At each orientation,200 independent images were generated for emitter within water locatedat h=700 nm with 2500 signal photons in total and 3 background photonsper pixel detected.

FIG. 48G is a set of graphs of the precision and accuracy of estimatingL−L₀ (nm) for emitters at h=700 nm with a nominal focal plane placed atz=−580 nm after filtering biased estimates at various orientations(Ω∈{0, 1, 2} sr, θ∈[0°, 90°], ϕ∈[0°, 360°)). Estimates >90 nm away fromthe ground truth 2D position (x0, y0) are removed. At each orientation,200 independent images were generated for emitter within water locatedat h=700 nm with 2500 signal photons in total and 3 background photonsper pixel detected.

FIG. 48H is a set of graphs of the precision and accuracy of estimatingh−h₀ (nm) for emitters at h=700 nm with a nominal focal plane placed atz=−580 nm after filtering biased estimates at various orientations(Ω∈{0, 1, 2} sr, θ∈[0°, 90°], ϕ∈[0°, 360°)). Estimates >90 nm away fromthe ground truth 2D position (x0, y0) are removed. At each orientation,200 independent images were generated for emitter within water locatedat h=700 nm with 2500 signal photons in total and 3 background photonsper pixel detected.

FIG. 48J is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=0 sr. 200 independent images weregenerated for emitter within water located at h=700 nm with 2500 signalphotons in total and 3 background photons per pixel detected. Afterfiltering biased estimates at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)), estimates >90 nm away from the ground truth2D position (x0, y0) are removed.

FIG. 48K is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=1 sr. 200 independent images weregenerated for emitter within water located at h=700 nm with 2500 signalphotons in total and 3 background photons per pixel detected. Afterfiltering biased estimates at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)), estimates >90 nm away from the ground truth2D position (x0, y0) are removed.

FIG. 48L is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=2 sr. 200 independent images weregenerated for emitter within water located at h=700 nm with 2500 signalphotons in total and 3 background photons per pixel detected. Afterfiltering biased estimates at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)), estimates >90 nm away from the ground truth2D position (x0, y0) are removed.

FIG. 49A is a set of graphs of the precision and accuracy of estimatingσ_(δ) (deg) for emitters at h=0 nm with a nominal focal plane placed atz=−580 nm with high background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=0 nm with 2500signal photons in total and 10 background photons per pixel detected.

FIG. 49B is a set of graphs of the precision and accuracy of estimatingσ_(Ω) (sr) for emitters at h=0 nm with a nominal focal plane placed atz=−580 nm with high background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=0 nm with 2500signal photons in total and 10 background photons per pixel detected.

FIG. 49C is a set of graphs of the precision and accuracy of estimatingσ_(L) (nm) for emitters at h=0 nm with a nominal focal plane placed atz=−580 nm with high background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=0 nm with 2500signal photons in total and 10 background photons per pixel detected.

FIG. 49D is a set of graphs of the precision and accuracy of estimatingδ_(h) (nm) for emitters at h=0 nm with a nominal focal plane placed atz=−580 nm with high background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=0 nm with 2500signal photons in total and 10 background photons per pixel detected.

FIG. 49E is a set of graphs of the precision and accuracy of estimatingδ−δ₀ (deg) for emitters at h=0 nm with a nominal focal plane placed atz=−580 nm with high background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=0 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 49F is a set of graphs of the precision and accuracy of estimatingΩ−Ω₀ (sr) for emitters at h=0 nm with a nominal focal plane placed atz=−580 nm with high background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=0 nm with 2500signal photons in total and 10 background photons per pixel detected.

FIG. 49G is a set of graphs of the precision and accuracy of estimatingL−L₀ (nm) for emitters at h=0 nm with a nominal focal plane placed atz=−580 nm with high background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=0 nm with 2500signal photons in total and 10 background photons per pixel detected.

FIG. 49H is a set of graphs of the precision and accuracy of estimatingh−h₀ (nm) for emitters at h=0 nm with a nominal focal plane placed atz=−580 nm with high background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=0 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 49J is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=0 sr. 200 independent images weregenerated for emitter within water located at h=0 nm with 2500 signalphotons in total and 10 background photons per pixel detected.

FIG. 49K is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=1 sr. 200 independent images weregenerated for emitter within water located at h=0 nm with 2500 signalphotons in total and 10 background photons per pixel detected.

FIG. 49L is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=2 sr. 200 independent images weregenerated for emitter within water located at h=0 nm with 2500 signalphotons in total and 10 background photons per pixel detected.

FIG. 50A is a set of graphs of the precision and accuracy of estimatingσ_(δ) (deg) for emitters at h=400 nm with a nominal focal plane placedat z=−580 nm with high background at various orientations (Ω∈{0, 1, 2}sr, θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independentimages were generated for emitter within water located at h=400 nm with2500 signal photons in total and 10 background photons per pixeldetected.

FIG. 50B is a set of graphs of the precision and accuracy of estimatingσ_(Ω) (sr) for emitters at h=400 nm with a nominal focal plane placed atz=−580 nm with high background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=400 nm with 2500signal photons in total and 10 background photons per pixel detected.

FIG. 50C is a set of graphs of the precision and accuracy of estimatingσ_(L) (nm) for emitters at h=400 nm with a nominal focal plane placed atz=−580 nm with high background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=400 nm with 2500signal photons in total and 10 background photons per pixel detected.

FIG. 50D is a set of graphs of the precision and accuracy of estimatingδ_(h) (nm) for emitters at h=400 nm with a nominal focal plane placed atz=−580 nm with high background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=400 nm with 2500signal photons in total and 10 background photons per pixel detected.

FIG. 50E is a set of graphs of the precision and accuracy of estimatingδ−δ₀ (deg) for emitters at h=400 nm with a nominal focal plane placed atz=−580 nm with high background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=400 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 50F is a set of graphs of the precision and accuracy of estimatingΩ−Ω₀ (sr) for emitters at h=400 nm with a nominal focal plane placed atz=−580 nm with high background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=400 nm with 2500signal photons in total and 10 background photons per pixel detected.

FIG. 50G is a set of graphs of the precision and accuracy of estimatingL−L₀ (nm) for emitters at h=400 nm with a nominal focal plane placed atz=−580 nm with high background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=400 nm with 2500signal photons in total and 10 background photons per pixel detected.

FIG. 50H is a set of graphs of the precision and accuracy of estimatingh−h₀ (nm) for emitters at h=400 nm with a nominal focal plane placed atz=−580 nm with high background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=400 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 50J is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=0 sr. 200 independent images weregenerated for emitter within water located at h=400 nm with 2500 signalphotons in total and 10 background photons per pixel detected.

FIG. 50K is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=1 sr. 200 independent images weregenerated for emitter within water located at h=400 nm with 2500 signalphotons in total and 10 background photons per pixel detected.

FIG. 50L is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=2 sr. 200 independent images weregenerated for emitter within water located at h=400 nm with 2500 signalphotons in total and 10 background photons per pixel detected.

FIG. 51A is a set of graphs of the precision and accuracy of estimatingσ_(δ) (deg) for emitters at h=700 nm with a nominal focal plane placedat z=−580 nm with high background at various orientations (Ω∈{0, 1, 2}sr, θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independentimages were generated for emitter within water located at h=700 nm with2500 signal photons in total and 10 background photons per pixeldetected.

FIG. 51B is a set of graphs of the precision and accuracy of estimatingσ_(Ω) (sr) for emitters at h=700 nm with a nominal focal plane placed atz=−580 nm with high background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=700 nm with 2500signal photons in total and 10 background photons per pixel detected.

FIG. 51C is a set of graphs of the precision and accuracy of estimatingσ_(L) (nm) for emitters at h=700 nm with a nominal focal plane placed atz=−580 nm with high background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=700 nm with 2500signal photons in total and 10 background photons per pixel detected.

FIG. 51D is a set of graphs of the precision and accuracy of estimatingδ_(h) (nm) for emitters at h=700 nm with a nominal focal plane placed atz=−580 nm with high background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=700 nm with 2500signal photons in total and 10 background photons per pixel detected.

FIG. 51E is a set of graphs of the precision and accuracy of estimatingδ-δ₀ (deg) for emitters at h=700 nm with a nominal focal plane placed atz=−580 nm with high background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=700 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 51F is a set of graphs of the precision and accuracy of estimatingΩ−Ω₀ (sr) for emitters at h=700 nm with a nominal focal plane placed atz=−580 nm with high background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=700 nm with 2500signal photons in total and 10 background photons per pixel detected.

FIG. 51G is a set of graphs of the precision and accuracy of estimatingL−L₀ (nm) for emitters at h=700 nm with a nominal focal plane placed atz=−580 nm with high background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=700 nm with 2500signal photons in total and 10 background photons per pixel detected.

FIG. 51H is a set of graphs of the precision and accuracy of estimatingh−h₀ (nm) for emitters at h=700 nm with a nominal focal plane placed atz=−580 nm with high background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). At each orientation, 200 independent imageswere generated for emitter within water located at h=700 nm with 2500signal photons in total and 3 background photons per pixel detected.

FIG. 51J is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=0 sr. 200 independent images weregenerated for emitter within water located at h=700 nm with 2500 signalphotons in total and 10 background photons per pixel detected.

FIG. 51K is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=1 sr. 200 independent images weregenerated for emitter within water located at h=700 nm with 2500 signalphotons in total and 10 background photons per pixel detected.

FIG. 51L is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=2 sr. 200 independent images weregenerated for emitter within water located at h=700 nm with 2500 signalphotons in total and 10 background photons per pixel detected.

FIG. 52A is a set of graphs of the precision and accuracy of estimatingσ_(δ) (deg) for emitters at h=400 nm with a nominal focal plane placedat z=−580 nm with low background at various orientations (Ω∈{0, 1, 2}sr, θ∈[0°, 90°], ϕ∈[0°, 360°)). An incorrect NFP position of z=−400 nmis used for estimation. At each orientation, 200 independent images weregenerated for emitter within water located at h=400 nm with 2500 signalphotons in total and 3 background photons per pixel detected.

FIG. 52B is a set of graphs of the precision and accuracy of estimatingσ_(Ω) (sr) for emitters at h=400 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). An incorrect NFP position of z=−400 nm isused for estimation. At each orientation, 200 independent images weregenerated for emitter within water located at h=400 nm with 2500 signalphotons in total and 3 background photons per pixel detected.

FIG. 52C is a set of graphs of the precision and accuracy of estimatingδ_(L) (nm) for emitters at h=400 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). An incorrect NFP position of z=−400 nm isused for estimation. At each orientation, 200 independent images weregenerated for emitter within water located at h=400 nm with 2500 signalphotons in total and 3 background photons per pixel detected.

FIG. 52D is a set of graphs of the precision and accuracy of estimatingδ_(h) (nm) for emitters at h=400 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], ϕ∈[0°, 360°)). An incorrect NFP position of z=−400 nm isused for estimation. At each orientation, 200 independent images weregenerated for emitter within water located at h=400 nm with 2500 signalphotons in total and 3 background photons per pixel detected.

FIG. 52E is a set of graphs of the precision and accuracy of estimatingδ-δ₀ (deg) for emitters at h=400 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). An incorrect NFP position of z=−400 nm isused for estimation. At each orientation, 200 independent images weregenerated for emitter within water located at h=400 nm with 2500 signalphotons in total and 3 background photons per pixel detected.

FIG. 52F is a set of graphs of the precision and accuracy of estimatingΩ−Ω₀ (sr) for emitters at h=400 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). An incorrect NFP position of z=−400 nm isused for estimation. At each orientation, 200 independent images weregenerated for emitter within water located at h=400 nm with 2500 signalphotons in total and 3 background photons per pixel detected.

FIG. 52G is a set of graphs of the precision and accuracy of estimatingL−L₀ (nm) for emitters at h=400 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). An incorrect NFP position of z=−400 nm isused for estimation. At each orientation, 200 independent images weregenerated for emitter within water located at h=400 nm with 2500 signalphotons in total and 3 background photons per pixel detected.

FIG. 52H is a set of graphs of the precision and accuracy of estimatingh−h₀ (nm) for emitters at h=400 nm with a nominal focal plane placed atz=−580 nm with low background at various orientations (Ω∈{0, 1, 2} sr,θ∈[0°, 90°], θ∈[0°, 360°)). An incorrect NFP position of z=−400 nm isused for estimation. At each orientation, 200 independent images weregenerated for emitter within water located at h=400 nm with 2500 signalphotons in total and 3 background photons per pixel detected.

FIG. 52J is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=0 sr. 200 independent images weregenerated for emitter within water located at h=400 nm with 2500 signalphotons in total and 3 background photons per pixel detected. Anincorrect NFP position of z=−400 nm is used for estimation.

FIG. 52K is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=1 sr. 200 independent images weregenerated for emitter within water located at h=400 nm with 2500 signalphotons in total and 3 background photons per pixel detected. Anincorrect NFP position of z=−400 nm is used for estimation.

FIG. 52L is a graph representing the percentage of trials thatsuccessfully detect one emitter for Ω=2 sr. 200 independent images weregenerated for emitter within water located at h=400 nm with 2500 signalphotons in total and 3 background photons per pixel detected. Anincorrect NFP position of z=−400 nm is used for estimation.

FIG. 53A is a graph of the axial position estimation h vs. polar angleestimation θ for an emitter in water with a nominal focal plane placedat z=−580 nm. The emitter here is located at (x₀, y₀, h₀)=(0, 0, 700) nmwith 3D orientation (θ₀, ϕ₀, Ω₀)=(38°, 162°, 0). The algorithm describedherein is used to fit 200 independently generated noisy images.

FIG. 53B is a graph of the horizontal position estimation x vs. polarangle estimation θ for an emitter in water with a nominal focal planeplaced at z=−580 nm. The emitter here is located at (x₀, y₀, h₀)=(0, 0,700) nm with 3D orientation (θ₀, ϕ₀, Ω₀)=(38°, 162°, 0). The algorithmis used to fit 200 independently generated noisy images.

FIG. 53C is a set of (top) Noiseless DSF images and (bottom) images withPoisson shot noise for an emitter whose position and orientation matchthe ground truth. (Red) x-polarized, (blue) y-polarized. Colorbar:photons/pixel. Scalebar: 500 nm.

FIG. 53D is a set of (top) Noiseless DSF images and (bottom) images withPoisson shot noise for an emitter whose position and orientation matchthe biased estimate shown in FIG. 53A and FIG. 53B. (Red) x-polarized,(blue) y-polarized. Colorbar: photons/pixel. Scalebar: 500 nm.

FIG. 53E is a histogram comparing the value of the negative loglikelihood l^(GT) _(NLL) of the ground truth (Eqn. S15) to the value ofthe negative log likelihood l^(est) _(NLL) of the biased position andorientation (>90 nm away from the ground truth 2D position (x₀, y₀)).Negative values indicate that the ground truth position and orientationare more consistent with the noisy image than the biased estimate, i.e.,l^(GT) _(NLL)<l^(est) _(NLL), but our estimation algorithm converged tothe biased value instead.

FIG. 54A is a back focal plane (BFP) images captured from thex-polarization channel using the flipping mirror in FIG. 30 . A binaryphase mask consisting of concentric rings of increasing radius (10-pixelinterval) is loaded onto the spatial light modulator (SLM). The centerof the phase mask is calibrated by matching the center of ring patternto the center of the objective lens pupil. Obtaining sharp images of thering pattern and pupil simultaneously ensures that the phase mask ispositioned properly along the optical axis. The BFPs shown here areilluminated by a single layer of fluorescent spheres on coverglassexposed to air. Scale bar: 500 μm.

FIG. 54B is a back focal plane (BFP) images captured from they-polarization channel using the flipping mirror in FIG. 30 . A binaryphase mask consisting of concentric rings of increasing radius (10-pixelinterval) is loaded onto the spatial light modulator (SLM). The centerof the phase mask is calibrated by matching the center of ring patternto the center of the objective lens pupil. Obtaining sharp images of thering pattern and pupil simultaneously ensures that the phase mask ispositioned properly along the optical axis. The BFPs shown here areilluminated by a single layer of fluorescent spheres on coverglassexposed to air. Scale bar: 500 μm.

FIG. 55A is an image of a perfect pixOL phase mask. Colorbar: phase(rad).

FIG. 55B is an image of a calibrated experimental pupil phase patternfrom the x-polarization channel. Colorbar: phase (rad).

FIG. 55C is an image of a calibrated experimental pupil phase patternfrom the y-polarization channel. Colorbar: phase (rad).

FIG. 55D is a set of images of ideal pixOL DSFs (red: x-polarized, blue:y-polarized) for various defocus positions z=−790 nm to z=610 nm. Theintensities of each red-blue image pair are normalized. Colorbar:normalized intensity. Scalebar: 400 nm.

FIG. 55E is a set of images of experimental DSFs (red: x-polarized,blue: y-polarized) for various defocus positions z=−790 nm to z=610 nm.The intensities of each red-blue image pair are normalized. Colorbar:normalized intensity. Scalebar: 400 nm.

FIG. 55F is a set of images of simulated DSFs (red: x-polarized, blue:y-polarized) for various defocus positions z=−790 nm to z=610 nm usingthe calibrated phase masks in FIG. 55B and FIG. 55C. The intensities ofeach red-blue image pair are normalized. Colorbar: normalized intensity.Scalebar: 400 nm.

FIG. 56A is an image of a perfect pixOL phase mask. Colorbar: phase(rad).

FIG. 56B is an image of a perfect conjugate pixOL phase mask (pixOL*).Colorbar: phase (rad).

FIG. 56C is an image of a calibrated experimental phase pattern from thex-polarization channel. Colorbar: phase (rad).

FIG. 56D is an image of a calibrated experimental phase pattern from they-polarization channel. Colorbar: phase (rad).

FIG. 56E is a set of ideal pixOL* DSFs (red: x-polarized, blue:y-polarized) for various defocus positions z=−790 nm to z=610 nm. Theintensities of each red-blue image pair are normalized. Colorbar:normalized intensity. Scalebar: 400 nm.

FIG. 56F is a set of experimental DSFs (red: x-polarized, blue:y-polarized) for various defocus positions z=−790 nm to z=610 nm. Theintensities of each red-blue image pair are normalized. Colorbar:normalized intensity. Scalebar: 400 nm.

FIG. 56G is a set of simulated DSFs using the calibrated phase masks inFIG. 56C and FIG. 56D (red: x-polarized, blue: y-polarized) for variousdefocus positions z=−790 nm to z=610 nm. The intensities of eachred-blue image pair are normalized. Colorbar: normalized intensity.Scalebar: 400 nm.

FIG. 57A is a set of images of the calibrated pixOL phase masks from the(i) x and (iii) y polarization channels. Smoothed versions of the pixOLphase mask are shown corresponding to DSFs in the (ii) x and (iv) ypolarization channels. The smoothed phase masks are reconstructed byprojecting the calibrated phase masks into a Zernike basis using thefirst 231 Zernike polynomials. Colorbar: phase (rad).

FIG. 57B is a set of images of the calibrated pixOL* phase masks fromthe (i) x and (iii) y polarization channels. Smoothed versions of thepixOL* phase mask are shown corresponding to DSFs in the (ii) x and (iv)y polarization channels. The smoothed phase masks are reconstructed byprojecting the calibrated phase masks into a Zernike basis using thefirst 231 Zernike polynomials. Colorbar: phase (rad).

FIG. 57C is a graph comparing model accuracy using the negative loglikelihood (NLL, Eqn. S15) between simulated dipole spread functions(DSFs) and images of fluorescent beads in FIG. 55D and FIG. 56E acrossall axial positions z. Differences in NLL are calculated for DSFs usingthe calibrated (NLL_(calibrated)) vs. smoothed phase masks(NLL_(smoothed)) for pixOL. Blue: NLL_(calibrated)−NLL_(smoothed);orange: NLL_(calibrated).

FIG. 57D is a graph comparing model accuracy using the negative loglikelihood (NLL, Eqn. S15) between simulated dipole spread functions(DSFs) and images of fluorescent beads in FIG. 55D and FIG. 56E acrossall axial positions z. Differences in NLL are calculated for DSFs usingthe calibrated (NLL_(calibrated)) vs. smoothed phase masks(NLL_(smoothed)) for pixOL*. Blue: NLL_(calibrated)−NLL_(smoothed);orange: NLL_(calibrated).

FIG. 58A is a set of image-plane basis images B of the pixOL phase maskfor an in-focus emitter of wavelength 641 nm. The image intensities arenormalized relative to the brightest basis image (B_(XX)).

FIG. 58B is a set of image-plane basis images B of the pixOL phase maskfor an in-focus emitter of wavelength 582 nm. The image intensities arenormalized relative to the brightest basis image (B_(XX)).

FIG. 58C is a set of the difference between basis images in FIG. 58A andFIG. 58B. Colorbars: normalized intensity. Scalebar: 500 nm.

FIG. 59A is a plot of the trajectory of defocus estimates z for a beadscanned axially from z=−790 nm to z=610 nm with a step size of 50 nm (11camera frames per step). Red dot: estimated axial distance z between thebead and focal plane in each frame; green cross: expected stageposition. Inset (i): Experimental axial precision σ_(z) at each scanningplane (mean precision σ_(z) is 3.51 nm). Inset (ii): Experimentalemission anisotropy precision σ_(Ω) at each scanning plane (averageprecision σ_(Ω) is 0.19 sr).

FIG. 59B is a plot of the trajectory of defocus estimates z for a beadscanned axially in the opposite direction of the bead in FIG. 59A fromz=−790 nm to z=610 nm with a step size of 50 nm (11 camera frames perstep). Red dot: estimated axial distance z between the bead and focalplane in each frame; green cross: expected stage position. Inset (i):Experimental axial precision σ_(z) at each scanning plane (meanprecision σ_(z) is 7.53 nm). Inset (ii): Experimental emissionanisotropy precision σ_(Ω) at each scanning plane (average precisionσ_(Ω) is 0.21 sr).

FIG. 59C is a graph of the mean emission anisotropies of the bead inFIG. 59A quantified as an effective wobble angle Ω at each scanningplane.

FIG. 59D is a graph of the mean emission anisotropies of the bead inFIG. 59B quantified as an effective wobble angle Ω at each scanningplane.

FIG. 59E is a graph of the mean emission anisotropies of the bead inFIG. 27E quantified as an effective wobble angle Ω at each scanningplane.

FIG. 60A is a graph of the photons s detected per localization of Nilered (NR) within spherical supported lipid bilayers (SLBs) consisting ofDPPC plus cholesterol. The red dashed line represents s=1000 photons.

FIG. 60B is a graph of the localization rate per 0.11 minutes for NRwithin DPPC plus cholesterol SLBs.

FIG. 60C is a graph of the photons s detected per localization of Nilered within spherical supported lipid bilayers (SLBs) consisting of DPPConly. The red dashed line represents s=1000 photons.

FIG. 60D is a graph of the localization rate per 0.11 minutes for NRwithin DPPC-only SLBs.

FIG. 61A is a set of SMOLM images of the 3D orientations and 3Dlocations of Nile red (NR) within a spherical supported lipid bilayerconsisting of DPPC plus cholesterol. Azimuthal angle ϕ of each NRmolecule for the three cross-sections in inset: (i) x-y slice, (ii) y-hslice, and (iii) plane with y+h=1000 nm. Each slice has a thickness of100 nm. Colorbar: ϕ (deg). (ii) and (iii) shares the same colormap shownin (iii). Scalebar: 400 nm.

FIG. 61B is an image of the NR orientations ϕ relative to theirpositions on the sphere's surface ϕ_(sphere). Colorbars: SM counts perbin.

FIG. 61C is an image of the NR axial locations h vs. orientations θ⊥.Colorbars: SM counts per bin. Red line: median value of θ⊥ at each axiallocation h.

FIG. 61D is a set of graphs of the experimental estimated bias inmeasuring NR (i) orientation θ⊥,_(bias) and (ii) wobble angle Ω_(bias),plotted as a function of NR position on the spherical surface[θ_(sphere), ϕ_(sphere)] of the SLB, assuming that the ground truthorientations of NR are normal to the spherical surface (θ⊥=0) andcompletely fixed (Ω=0 sr). (iii,iv) Same as (i,ii) but for (iii) angularstandard deviation σ_(δ) (Eqn. S23) and (iv) wobble angle precisionσ_(Ω).

FIG. 61E is a histogram of the NR wobble Ω for (green) all localizationsover the sphere and (purple) molecules within the membrane defect inFIG. 28E.

FIG. 61F is a set of images of (top row) sum of all pixOL images of NRlocated within a 50×50×100 nm³ box at three locations in FIG. 61A inset:(1) h=150 nm, (2) h=550 nm, (3) h=950 nm. (bottom row) Simulated pixOLimages for emitters oriented perpendicular to the SLB and centered atthe three dots locations in FIG. 61A inset with ϕ_(sphere)=135°. (Red)x-polarized, (blue) y-polarized. Colorbars: normalized intensity.Scalebar: 500 nm.

FIG. 61G is a set of histograms of NR lateral positions r (see FIG.61A(ii)) relative to the sphere's center within each h slice compared toa model distribution (yellow lines, Eqn. S27), accounting for the curvedspherical surface and pixOL's localization precision.

FIG. 62A is a set of SMOLM images of the 3D orientations and 3Dlocations of Nile red (NR) within a spherical supported lipid bilayerconsisting of DPPC only. Azimuthal angle ϕ of each NR molecule for threeslices in inset: (i) x-y slice, (ii) y-h slice, and (iii) plane withy+h=1000 nm. Each slice has a thickness of 100 nm. Colorbar: ϕ (deg).(ii) and (iii) shares the same colormap shown in (iii). Scale bar: 400nm.

FIG. 62B is an image of NR orientations ϕ relative to their positions onthe sphere's surface ϕ_(sphere) for estimations with signal photonslarger than 1000.

FIG. 62C is an image of NR axial locations h vs. orientations θ⊥ forestimations with signal photons larger than 1000. Colorbars: SM countsper bin. Red line: the median value of θ⊥ at each axial location h.

FIG. 62D is a set of images of x-y cross-sections of the bead depictingthe 3D orientation (θ, ϕ) of each NR as a line segment. The length anddirection of each line indicate in-plane magnitude (μ_(x) ²+μ_(y) ²)1/2and azimuthal orientation ϕ, respectively. Colors represent azimuthalorientation ϕ. Scale bar: 400 nm

FIG. 62E is a set of histograms NR lateral positions r (see FIG.62A(ii)) relative to the sphere's center within each h slice compared toa model distribution (yellow lines, Eqn. S27), accounting for the curvedspherical surface and pixOL's localization precision.

FIG. 62F is an image of NR azimuthal orientations ϕ relative to theirsignal photons.

FIG. 62G is an image of NR azimuthal orientations ϕ relative to theirwobble angles Ω.

FIG. 62H is an image of NR wobble angle Ω relative to their signalphotons.

FIG. 63A is a computer algorithm for estimating the 3D orientation and3D location of individual emitters based on cropped images.

FIG. 63B is a computer algorithm for estimating 3D orientation and 3Dlocation using FISTA with backtracking.

There are shown in the drawings arrangements that are presentlydiscussed, it being understood, however, that the present embodimentsare not limited to the precise arrangements and are instrumentalitiesshown. While multiple embodiments are disclosed, still other embodimentsof the present disclosure will become apparent to those skilled in theart from the following detailed description, which shows and describesillustrative aspects of the disclosure. As will be realized, theinvention is capable of modifications in various aspects, all withoutdeparting from the spirit and scope of the present disclosure.Accordingly, the drawings and detailed description are to be regarded asillustrative in nature and not restrictive.

DETAILED DESCRIPTION

A defining feature of soft matter is the impact of thermal fluctuationson the organization and self-assembly of molecules into mesoscopicstructures like lipid membranes—processes that are notoriously difficultto observe directly. SMOLM extends conventional SMLM to measure both thepositions and 3D orientations of >105 single fluorescent molecules withhigh precision in under 5 minutes. The orientation and rotationaldynamics of fluorescent probes reveal their interactions with thesurrounding environment, such as the ordering of and condensationdynamics within lipid membranes. Nanoscale imaging spectroscopy measuressingle-molecule orientation spectra, i.e., the six orientational secondmoments of dipole emitters, to resolve nanoscale chemical properties,similar to classic spectroscopies such as absorption, fluorescenceemission, fluorescence lifetime, and NMR.

Single-molecule orientation localization microscopy, also referred toherein as SMOLM, directly measures the orientation spectra (3Dorientation plus “wobble”) of lipophilic probes transiently bound tolipid membranes. SMOLM sheds light on the organizational and functionaldynamics of lipid-lipid, lipid-protein, and other soft matter assembliesat the single-molecule level with nanoscale resolution.

To achieve high spatiotemporal resolution and sufficient samplingdensity, the PAINT (points accumulation for imaging in nanoscaletopography) blinking mechanism may be applied, in which certainlipophilic dyes exhibit fluorescence solely while in a non-polarenvironment. We thereby resolve nanoscale lipid domains with resolutionbeyond the diffraction limit and monitor in-situ lipid compositionalchanges induced by low doses of sphingomyelinase. SMOLM imaging clearlyshows its potential to resolve interactions between various lipidmolecules, enzymes, and fluorescent probes with detail that has neverbeen achieved previously. SMOLM imaging makes use of anorientation-sensitive engineered dipole spread function (DSF) toefficiently encode the 3D orientation and wobble of dipole-like emittersinto fluorescence images, as well as a maximum likelihood estimator thatpromotes sparsity for estimating molecular position, orientation, andwobble from those images robustly and accurately. This combination ofhardware and software is critical for resolving molecular positions andorientations unambiguously and accurately; otherwise, neighboringmolecules, wobbling molecules, and translationally diffusing moleculescould be confused with one another.

To improve imaging sensitivity, DSF engineering makes use of a phasemask at the back focal plane of a microscope to modulate the detectedDSFs of each emitter (see FIG. 2 ). Applications that have a lowsignal-to-noise ratio (SNR) or densely labeled samples pose challengesto existing multi-spot DSFs. It is desirable to optimize a DSF for aspecific sample of interest to achieve optimal imaging performance.

In various aspects, an algorithm, termed pixOL, simultaneously optimizesall pixels of a phase mask across two orthogonally polarized imagingchannels for applications with a low SNR. Unlike optimization usingZernike polynomial, pixOL can directly take advantage of super-criticalfluorescence arising from imaging SMs near a refractive index interface.Using the resulting phase mask provides for simultaneous measurement of3D orientation and 3D location of an emitter with small statisticalvariance. The parameters in the pixOL algorithm may be varied tooptimize a phase mask to fit a range of microscopes and imagingapplications.

As described in the example below, the pixOL DSF may be used to measurethe orientation of Nile red (NR) molecules transiently attached to twosupported lipid bilayers (SLBs): DPPC (di(16:0) PC) and DPPC withcholesterol. The orientations of NR measured by the pixOL DSF reveal thechemical compositions of the SLBs.

Localization and Orientation Estimation of SMOLM

Imaging Model and Pixol Algorithm

In one aspect, the goal of the pixOL algorithm is to simultaneouslyoptimize all pixels of a phase mask P∈R^(n×n) so that the resultingdipole-spread function (DSF) has optimal precision for measuring the 3Dorientation [μ_(x), μ_(y), μ_(z), Ω] of dipole-like emitters. Thedipole's average orientation (θ, ϕ) in spherical coordinates during acamera frame can be related to its unit vector parameterizationμ=[μ_(x), μ_(y), μ_(z)]^(T) in Cartesian space by

[μ_(x),μ_(y),μ_(z)]=[sin θ cos ϕ, sin θ sin ϕ, cos θ].  (S1)

A wobbling dipole with the orientation parameters [μ_(x), μ_(y), μ_(z),Ω], where the solid angle Ω characterizes rotational diffusion within ahard-edged cone, may be characterized using the six orientational secondmoments m∈R⁶ as

m=

μ _(x) ²

,

μ_(y) ²

,

μ_(z) ²

,

μ_(x)μ_(y)

,

μ_(x)μ_(z)

,

μ_(y)μ_(z)

^(T)  (S2)

where

μ_(x) ²

=γμ_(x) ²+(1−γ)/3,  (S3 a)

μ_(y) ²

=γμ_(y) ²+(1−γ)/3,  (S3b)

μ_(z) ²

=γμ_(z) ²+(1−γ)/3,  (S3c)

μ_(x)μ_(y)

γμ_(x)μ_(y),  (S3d)

μ_(x)μ_(z)

=γμ_(x)μ_(z),  (S3e)

μ_(y)μ_(z)

=γμ_(y) μz,  (S3f)

γ=1−3Ω/4π+Ω²/8π².  (S3g)

In some embodiments, the phase mask can be optimized for a microscopethat splits the collected fluorescence into x- and y-polarized channels(FIG. 26A). To model such a system, the x- and y-polarized electricfield E_(BFP) at the back focal plane (BFP) can be written as

$\begin{matrix}{{E_{BFP}\left( {u,v} \right)} = {{\exp\left( {{jk}_{1}z\sqrt{1 - \left( {u^{2} + v^{2}} \right)}} \right)}{\exp\left( {{jk}_{h}h\sqrt{1 - {\frac{n_{1}^{2}}{n_{h}^{2}}\left( {u^{2} + v^{2}} \right)}}} \right)}}} & ({S4})\end{matrix}$ ${\begin{bmatrix}{g_{x,{BFP}}^{(x)}\left( {u,v} \right)} & {g_{y,{BFP}}^{(x)}\left( {u,v} \right)} & {g_{z,{BFP}}^{(x)}\left( {u,v} \right)} \\{g_{x,{BFP}}^{(y)}\left( {u,v} \right)} & {g_{y,{BFP}}^{(y)}\left( {u,v} \right)} & {g_{z,{BFP}}^{(y)}\left( {u,v} \right)}\end{bmatrix}\begin{bmatrix}\mu_{x} \\\mu_{y} \\\mu_{x}\end{bmatrix}},$

Where g_(i,BFP) ^((l)) are the basis fields observed in an i-polarizedBFP from a dipole at axial position h with orientation μ_(i) and i,1∈{x, y, z}. The objective can be focused at a nominal focal planepositioned at a distance −z above the coverslip. The wavenumberk_(h)=nh2π/λ represents the wave propagation constant in a sample withrefractive index of n_(h) (assumed to be water in each of ourexperiments, n_(h)=1.33), and the wavenumber k₁=n₁2π/λ represents thewave propagation constant in immersion oil with refractive index ofn1=1.518. The BFP coordinates can be normalized such that u²+v²<1. Tooptimize both axial precision and orientation precision across a largeaxial range, the trade-offs between the multi-parameter optimizationgoals must be balanced. In a particular embodiment, the pixOL phase maskis optimized for in-focus emitters at the water-glass interface (z=h=0nm).

For computational convenience, a discrete model g_(i) of the basisfields can be used, sampling the BFP with sufficient pixels andzero-padding to match the spatial light modulator (SLM) of the imagingsystem. The polarized basis fields [g_(i) ^((x)), g_(i) ^((y))]^(T) canbe concatenated at the image plane of the microscope and express themjointly as

$\begin{matrix}{{g_{i} = {\begin{bmatrix}g_{i}^{(x)} \\g_{i}^{(y)}\end{bmatrix} = {\mathcal{F}\left\{ {\begin{bmatrix}{\exp({jP})} \\{\exp({jP})}\end{bmatrix} \odot \begin{bmatrix}g_{i,{BFP}}^{(x)} \\g_{i,{BFP}}^{(y)}\end{bmatrix}} \right\}}}},} & ({S5})\end{matrix}$

where F{·} denotes a discrete 2D Fourier transform, ⊙ representselement-wise multiplication of two vectors or matrices. The basis imagesB=[B_(xx), B_(yy), B_(zz), B_(xy), B_(xz), B_(yz)]∈R^(N×6) in the imageplane can be calculated as

B _(xx) =g _(x) ⊙g _(x)*  (S6a)

B _(yy) =g _(y) ⊙g _(y)*  (S6b)

B _(zz) =g _(z) ⊙g _(z)*  (S6c)

B _(xy) =g _(x) ⊙g _(y) *+g _(x) *⊙g _(y)  (S6d)

B _(xz) =g _(x) ⊙g _(z) *+g _(x) *⊙g _(z)  (S6e)

B _(yz) =g _(y) ⊙g _(z) *+g _(y) *⊙g _(z).  (S6f)

These images B_(il) correspond to the DSF (Eqn. 1) produced by a dipolewith an orientational second moment m_(il), where i,l∈{x, y, z}. Asimilar expression can be derived for the basis images B_(BFP) in theback focal plane (FIG. 31 ).

In some embodiments, the best possible precision for estimating theorientational moments m can be calculated using the Cramér-Rao bound(CRB) matrix K using

$\begin{matrix}{{K = \left( {\sum\limits_{j = 1}^{N}{\frac{s^{2}}{I_{j}}B_{j}^{T}B_{j}}} \right)^{- 1}},} & ({S7})\end{matrix}$

where B_(j)∈R^(1×6) is the jth row of B, the superscript T denotes amatrix transpose.

In various aspects, the pixOL method and SMOLM imaging methods make useof an emitter model characterizing the physical and optical propertiesof the emitters to be imaged using the SMOLM system as described herein.In various aspects, the emitter model may be representative of anysuitable emitter type including, but not limited to, a dipole-likeemitter, as described below.

In various aspects, a marker model typically used in SMOLM imagingincluding, but not limited to, a fluorescent molecule may be modeled asa dipole-like emitter wobbling within a cone (see FIG. 1 ). Anorientational unit vector μ=[μ_(x), μ_(y), μz]=[sin θ cos ϕ, sin θ sinϕ, cos θ]T and a solid angle Ω define the center orientation and thewobbling area of the cone, respectively. Assuming that a molecule'srotational correlation time is faster than its excited state lifetimeand the camera acquisition time, its orientation state can be fullcharacterized by a second-moment vector (see FIG. 4 ) m=[

μ_(x) ²

,

μ_(y) ²

,

μ_(z) ²

,

μ_(x)μ_(y)

,

μ_(x)μ_(z)

,

μ_(y)μ_(z)

,]^(T) where each component is a time-averaged second moment of μ withina single camera acquisition period. A fluorescence microscope image ofsuch an emitter captured by an n-pixel camera I∈

^(n), can be modeled as a linear superposition of six basis imagesweighted by m as follows:

I=sBm+b=s[B _(xx) ,B _(yy) ,B _(zz) ,B _(xy) ,B _(xz) ,B _(yz)]m+b,  (1)

where s is the number of photons detected from the molecule and b is thenumber of background photons in each pixel. Referring to FIG. 3 , eachbasis image B_(k)∈

^(n) (k∈{xx, yy, zz, xy, xz, yz}) corresponds to the response of theoptical system to each orientational second-moment component m_(k) andcan be calculated by the vectorial diffraction theory.

Single-Molecule Detection and 3D Orientation and 3D Position Estimation

Detecting single molecules (SMs) and estimating their 3D positions andorientations (6 second moments m) can be computationally expensive andnon-convex. In some embodiments, these challenges can be overcome by 1)applying a linear approximation to the forward imaging model and 2)separating the detection and estimation process into sequential steps.

In some embodiments, the forward model (Eqn. 1) can be extended toaccommodate images containing Q≥1 SMs such that

$\begin{matrix}{{I = {{\sum\limits_{q}{s_{q}{B\left( {x_{q},y_{q}} \right)}m_{q}}} + b}},} & ({S9})\end{matrix}$

where s_(q) is the brightness in photons, [x_(q), y_(q), h_(q)] is the3D location, and m_(q) is the orientational second moment vector of theqth emitter.

In some embodiments, the basis images B change shape for SMs atdifferent axial locations, but only shifts linearly for SMs at differentlateral positions. To reduce the computational burden of 3Dlocalization, the continuous 3D location space can be discretized usinga first-order polynomial approximation. The imaging model becomes

$\begin{matrix}{{I = {{\sum\limits_{q}\left\lbrack {{s_{q}{B\left( {x_{q0},y_{q0},h_{q0}} \right)}m_{q}} + {s_{q}\frac{\partial{B\left( {x_{q0},y_{q0},h_{q0}} \right)}}{\partial x}m_{q}{dx}_{q}s_{q}\frac{\partial{B\left( {x_{q0},y_{q0},h_{q0}} \right)}}{\partial y}m_{q}{dy}_{q}} + {s_{q}\frac{\partial\left( {x_{q0},y_{q0},h_{q0}} \right)}{\partial h}m_{q}{dh}_{q}}} \right\rbrack} + b}},} & ({S10})\end{matrix}$

where [x_(q0), y_(q0), h_(q0)] is the closest discrete grid point to thecontinuous location [x_(q), y_(q), h_(q)]. The off-grid distances[dx_(q), dy_(q), dh_(q)]=[x_(q), y_(q), h_(q)]−[x_(q0), y_(q0), h_(q0)]characterize the difference between the true position and the closestgrid point. As the last three columns (i.e., images) of the basis matrixB have a total energy of zero (FIG. 35 ), the forward model can befurther simplified by excluding these last three images in thefirst-order approximation. The forward model becomes

$\begin{matrix}{{I = {{\sum\limits_{q}{A_{q}\zeta_{q}}} + b}},} & ({S11})\end{matrix}$

where

$\begin{matrix}{{A_{q} = \left\lbrack {{B\left( {x_{q0},y_{q0},h_{q0}} \right)},\frac{\partial{B^{o}\left( {x_{q0},y_{q0},h_{q0}} \right)}}{\partial x},\frac{\partial{B^{o}\left( {x_{q0},y_{q0},h_{q0}} \right)}}{\partial y},\frac{\partial{B^{o}\left( {x_{q0},y_{q0},h_{q0}} \right)}}{\partial h}} \right\rbrack},} & ({S12a})\end{matrix}$ $\begin{matrix}{{\zeta_{q} = \left\lbrack {{s_{q}m_{q}^{T}},{{s_{q}\left( m_{q}^{o} \right)}^{T}{dx}_{q}},{{s_{q}\left( m_{q}^{o} \right)}^{T}{dy}_{q}},{{s_{q}\left( m_{q}^{o} \right)}^{T}{dh}_{q}}} \right\rbrack^{T}},} & ({S12b})\end{matrix}$

and the basis matrix B_(o) excludes the last three basis images (FIG. 39) and the second moment vector m_(oq) excludes the last three elements.Importantly, the matrix A_(q) may be precomputed for a specific imagingsystem and choice of location grid, while ζ_(q) contains the molecularbrightness, orientation, and position information to be estimated. In aparticular embodiment, the grid spacing can be set to [p_(x)=58.5,p_(y)=58.5, p_(h)=50] nm along the x, y, and h directions. With thisgrid size, the approximated pixOL DSF can be very similar to that usinga fully accurate model (FIG. 40 ). The chosen grid size balancescomputational speed and accuracy; in other embodiments, approximationerrors can be reduced by using smaller grid sizes at the cost ofcomputational burden. In these embodiments, the 3D orientation and 3Dposition estimation problem can be simplified using a linear forwardmodel (Eqn. S11) involving only 15 elements in ζ_(q) for each emitter,i.e., 6 brightness-weighted second moments at each spatial grid pointplus 3 brightness- and moment-weighted first-order distances for eachoff-grid direction. In other embodiments, the original first-orderapproximation model in Eqn. S10 involves 24 parameters for each SM.

In some embodiments, detection and estimation can be performed asseparate tasks involving three main parts. In these embodiments,determining the number of emitters and their 2D locations within animage. In the second step, an initial estimate of an emitter's 3Dorientation and 3D position is obtained based on cropped images centeredat each detected SM. Finally, the algorithm updates the estimates of allemitters simultaneously using the entire image.

In some embodiments, to simplify the detection process, the axialdimension can be ignored and a 2D forward model is used

$\begin{matrix}{{I^{2D} = {{\sum\limits_{q}{\zeta_{q}^{2D}A_{q}^{2D}}} + b}},} & ({S13})\end{matrix}$

where

$\begin{matrix}{{A_{q}^{2D} = \left\lbrack {{B_{a}\left( {x_{q0},y_{q0}} \right)},\frac{\partial{B_{a}^{o}\left( {x_{q0},y_{q0}} \right)}}{\partial x},\frac{\partial{B_{a}^{o}\left( {x_{q0},y_{q0}} \right)}}{\partial y}} \right\rbrack},} & ({S14a})\end{matrix}$ $\begin{matrix}{{\zeta_{q}^{2D} = \left\lbrack {{s_{q}m_{q}^{T}},{{s_{q}\left( m_{q}^{o} \right)}^{T}{dx}_{q}},{{s_{q}\left( m_{q}^{o} \right)}^{T}{dy}_{q}}} \right\rbrack^{T}},} & ({S14b})\end{matrix}$ $\begin{matrix}{{B_{a} = {\left( {{B\left( {x_{q},y_{q},{h_{q} = 0}} \right)} + {B\left( {x_{q},y_{q},{h_{q} = {900{nm}}}} \right)}} \right)/2}},} & ({S14c})\end{matrix}$

and B_(a) is an average basis matrix combining the basis images at thetop of the focal volume with the basis images at the bottom axial plane(FIG. 41 ). A RoSE-O algorithm can be used, which is designed to detectand estimate the 2D locations and 3D orientations of SMs, to determinethe number of emitters and an initial estimate of their 2D locations(x_(q), y_(q)).

In some embodiments, the input image to RoSE-O can be concatenated fromcropped images collected from the x and y polarization channels. Inthese embodiments, fluorescent beads can be used to generate apolynomial registration map between the two channels. Then the center ofthe field of view of interest in the y polarization channel can bemanually selected. The corresponding center in the x channel can becalculated using the registration map. Then two images with a smallfield of view centered at the chosen position within the twopolarization channels can be cropped, and the two images can beconcatenated together to form a single image. This image can be used byRoSE-O to detect SMs and estimate each of their 2D positions and 3Dorientations (FIG. 42 ).

In the second step, an initial estimate of the 3D orientations and 3Dlocations of individual emitters can be obtained. For each detectedemitter, an image of 21×21 pixels centered at the grid point (x_(q0),y_(q0)) nearest to the estimated 2D location (x_(q), y_(q)) from thefirst step can be isolated. Using each cropped image I_(det), algorithm1 in FIG. 63A can estimate the 15 parameters in ζ_(q) simultaneously byminimizing the negative log-likelihood for qth emitter

$\begin{matrix}{\ell_{{NLL},q} = {\sum\limits_{i = 1}^{N}{\left\lbrack {{A_{q}\zeta_{q}} + b - {I_{\det} \odot {\log\left( {{A_{q}\zeta_{q}} + b} \right)}}} \right\rbrack_{i}.}}} & ({S15})\end{matrix}$

where [⋅]i represents the ith element of a vector. In step 3, theestimated ζq for the Q SMs can be combined as the initial value ζ₀.Algorithm 2 in FIG. 63B, based on FISTA, can be used to refinebrightness, position, and orientation estimates ζq simultaneously forall emitters based on the entire captured image by minimizing thenegative log-likelihood for all Q SMs

$\begin{matrix}{\ell_{NLL} = {\sum\limits_{i}^{N}{\left\{ {{\sum\limits_{q}\left( {A_{q}\zeta_{q}} \right)} + b - {I_{\det} \odot {\log\left\lbrack {{\sum\limits_{q}\left( {A_{q}\zeta_{q}} \right)} + b} \right\rbrack}}} \right\}_{i}.}}} & ({S16})\end{matrix}$

The 3D locations (x_(q), y_(q), h_(q)) and orientational momentsm^({circumflex over ( )}) _(q) of molecule q can be extracted from ζ_(q)using

$\begin{matrix}{{{\hat{s}}_{q} = {\sum\limits_{i = 1}^{3}\zeta_{q,i}}},} & ({S17a})\end{matrix}$ $\begin{matrix}{{{\hat{m}}_{q} = {\frac{1}{{\hat{s}}_{q}}\zeta_{q,{1:6}}}},} & ({S17b})\end{matrix}$ $\begin{matrix}{{{\hat{x}}_{q} = {{\hat{x}}_{q0} + \frac{\sum_{i = 7}^{9}\zeta_{q,i}}{\sum_{i = 7}^{3}\zeta_{q,i}}}},} & ({S17c})\end{matrix}$ $\begin{matrix}{{{\hat{y}}_{q} = {{\hat{y}}_{q0} + \frac{\sum_{i = 10}^{12}\zeta_{q,i}}{\sum_{i = 7}^{3}\zeta_{q,i}}}},} & ({S17d})\end{matrix}$ $\begin{matrix}{{{\hat{h}}_{q} = {{\hat{h}}_{q0} + \frac{\sum_{i = 13}^{15}\zeta_{q,i}}{\sum_{i = 7}^{3}\zeta_{q,i}}}},} & ({S17e})\end{matrix}$

where ζ_(q,i) represents the ith element of vector ζ_(q). The estimatedsecond-moment vectors m^({circumflex over ( )}) _(q) are next projectedto first-moment orientation space (^({circumflex over ( )})ϕ,^({circumflex over ( )})θ, ^({circumflex over ( )})Ω) using a weightedleast-square estimator as follows.

$\begin{matrix}{\left( {\hat{\phi},\hat{\theta},\hat{\Omega}} \right) = {\underset{\phi^{\prime},\theta^{\prime},\Omega^{\prime}}{\arg\min}\left( {{\hat{m}}_{q} - {m_{q}\left( {\phi^{\prime},\theta^{\prime},\Omega^{\prime}} \right)}} \right)^{T}{{K^{- 1}\left( {{\hat{m}}_{q} - {m_{q}\left( {\phi^{\prime},\theta^{\prime},\Omega^{\prime}} \right)}} \right)}.}}} & ({S18})\end{matrix}$

Note that weighting by a Fisher information matrix (K⁻¹) ensures thatmore weight is given to the second moments m_(i) for which pixOLdemonstrates superior precision.

When estimating ζq, two sequential projection operations can be used toenhance the convexity and guarantee that the second-momentm^({circumflex over ( )}) _(q) estimates are physically meaningful. Thefirst projection P_(L1) enforces the first three brightness-weightedmoments [ζ_(q,1), ζ_(q,2), ζ_(q,3)] to be positive. It also ensures thatthe off-grid distances [d_(xq), d_(yq), d_(hq)] are smaller than athreshold t, since a large off-grid distance will reduce the robustnessof the first-order polynomial approximation (Eqn. S10). Parameters[ζ_(q,i), dw_(iq)] related to each orientational moment m_(q,I) areprojected separately, where

i∈{1,2,3},  (S19a)

dw _(q) ¹=ζ_(q,[7,10,13]),  (S19)

dw _(q) ²=ζ_(q,[8,11,14]),  (S19c)

dw _(q) ³=ζ_(q,[9,12,15]).  (S19d)

This projection can be written as

$\begin{matrix}{{P_{L1}\left( \left\lbrack {\zeta_{q,i},{dw}_{q}^{i}} \right\rbrack \right)} = \left\{ \begin{matrix}{0,} & {{{if}{{dw}_{q}^{i}}_{2}} \leq \frac{- \zeta_{q,i}}{t}} \\{\left( {\zeta_{q,i},{dw}_{q}^{i}} \right),} & {{{if}{{dw}_{q}^{i}}_{2}} \leq {\zeta_{q,i}t}} \\\begin{matrix}\left( {\frac{\zeta_{q,i} + {t{{dw}_{q}^{i}}_{2}}}{1 + t^{2}},} \right. \\{\left. {\frac{\zeta_{q,i} + {t{{dw}_{q}^{i}}_{2}}}{1 + t^{2}}\frac{{tdw}_{q}^{i}}{{{dw}_{q}^{i}}_{2}}} \right),}\end{matrix} & {{{{if}{{dw}_{q}^{i}}_{2}} \geq {\zeta_{q,i}t}},}\end{matrix} \right.} & ({S20})\end{matrix}$

and can be set to t=117 nm.

The second projection P_(L2) ensures that the second-momentm^({circumflex over ( )}) _(q) estimates correspond to a wobble Ω thatis physically meaningful, given by

P _(L2)(π_(q))=[π_(q,1:3) ,kπ _(q,4:6),π_(q,7:15)],  (S21)

where

$\begin{matrix}{{k = \frac{\max\left( {0,{\min\left( {{{1.5e} - 0.5},1} \right)}} \right)}{{1.5e} - 0.5}},} & ({S22})\end{matrix}$

and e is the largest eigenvalue of the second moment matrix.

The algorithm can be modified to estimate the NFP of the objective z ifthe axial positions h of the emitters are known (FIG. 27E and FIG. 59 ).If knowledge is lacking of both the emitter axial positions and the NFPposition, it is possible to “guess” an arbitrary NFP position. Thealgorithm will yield an estimated emitter axial position that willsimply be shifted by the difference between the hypothetical NFPposition and the true NFP position (FIG. 52 ).

In some aspects, the locations and orientations of single molecules maybe estimated simultaneously using a sparsity-promoting maximumlikelihood estimator. Briefly, the object space is represented by arectangular lattice of grid points with spacing equal to the camerapixel size. Each grid point may contain at most a single moleculeparameterized by brightness, position offsets, and six orientationalsecond moments.

To robustly estimate the number of underlying molecules and theirparameters in the presence of SM image overlap, a regularized maximumlikelihood exploiting a group-sparsity norm to estimate the parametersof each grid point may be used. The algorithm begins by estimating thestrength, i.e., brightness, of each of the second moments {tilde over(m)}_(k) independently at all object grid points. The localizations maybe pooled together, i.e., their brightnesses and position offsets,across the six second moments, to identify the most likely molecules inthe object space. Once these molecules are identified, a constrainedmaximum likelihood is solved to minimize systematic biases induced bythe sparsity norm, yielding estimates of the brightnesses, locations,and orientations (second moments {tilde over (m)}) of all molecules inthe image. Localizations with signal estimates of less than 400 photonsdetected may be removed to eliminate unreliable localizations.

The estimated second-moment vectors {tilde over (m)} were next projectedto the physical orientation space (polar angle θ, azimuthal angle ϕ, andwobbling area Ω of a transition dipole moment μ) by a weightedleast-square estimator:

$\begin{matrix}{\left( {\theta,\phi,\Omega} \right) = {\underset{\hat{\theta},\hat{\phi},\hat{\Omega}}{\arg\min}\left( {\overset{\sim}{m} - {m\left( {\hat{\theta},\hat{\phi},\hat{\Omega}} \right)}} \right)^{T}{{FIM}\left( {\overset{\sim}{m} - {m\left( {\hat{\theta},\hat{\phi},\hat{\Omega}} \right)}} \right)}}} & (2)\end{matrix}$

such that

$\begin{matrix}{{{m\left( {\theta,\phi,\Omega} \right)} = \left\lbrack {\left\langle \mu_{x}^{2} \right\rangle,\left\langle \mu_{y}^{2} \right\rangle,\left\langle \mu_{z}^{2} \right\rangle,\left\langle {\mu_{x}\mu_{y}} \right\rangle,\left\langle {\mu_{x}\mu_{z}} \right\rangle,\left\langle {\mu_{y}\mu_{z}} \right\rangle} \right\rbrack^{T}},} & (3)\end{matrix}$ $\begin{matrix}{{\left\langle \mu_{x}^{2} \right\rangle = {{\gamma\mu_{x}^{2}} + \frac{1 - \gamma}{3}}},{\left\langle {\mu_{x}\mu_{y}} \right\rangle = {\gamma\mu_{x}\mu_{y}}},} & (4)\end{matrix}$ $\begin{matrix}{{\left\langle \mu_{y}^{2} \right\rangle = {{\gamma\mu_{y}^{2}} + \frac{1 - \gamma}{3}}},{\left\langle {\mu_{x}\mu_{z}} \right\rangle = {\gamma\mu_{x}\mu_{z}}},} & (5)\end{matrix}$ $\begin{matrix}{{\left\langle \mu_{z}^{2} \right\rangle = {{\gamma\mu}_{z}^{2} + \frac{1 - \gamma}{3}}},{\left\langle {\mu_{y}\mu_{z}} \right\rangle = {\gamma\mu_{y}\mu_{z}}},} & (6)\end{matrix}$ $\begin{matrix}{{\left\lbrack {\mu_{x},\mu_{y},\mu_{z}} \right\rbrack = \left\lbrack {{\sin\theta\cos\phi},{\sin\theta\sin\phi},{\cos\theta}} \right\rbrack},{and}} & (7)\end{matrix}$ $\begin{matrix}{{\gamma = {{\frac{1}{2}{\cos^{2}\left( {2\arcsin\sqrt{\frac{\Omega}{8\pi}}} \right)}} + {\frac{1}{2}{\cos\left( {2\arcsin\sqrt{\frac{\Omega}{8\pi}}} \right)}}}},} & (8)\end{matrix}$

where γ is the rotational constraint and FIM is the Fisher information(FI) matrix calculated from the basis images. Here, the FI matrixassociated with estimating the six orientational second moments m isdefined as

$\begin{matrix}{{{FIM} = {\sum\limits_{i = 1}^{n}{\frac{1}{I_{i}}\bigtriangledown I_{i}^{T}}}},{\bigtriangledown I_{i}},} & (9)\end{matrix}$

where i denotes the ith pixel of an image I∈

^(n) captured by a camera and

${\bigtriangledown I_{i}} = {\left\lbrack {\frac{\partial I_{i}}{\partial m_{xx}},\frac{\partial I_{i}}{\partial m_{yy}},\frac{\partial I_{i}}{\partial m_{zz}},\frac{\partial I_{i}}{\partial m_{xy}},\frac{\partial I_{i}}{\partial m_{xz}},\frac{\partial I_{i}}{\partial m_{yz}}} \right\rbrack.}$

Due to the linearity of the forward imaging model (1) in terms of thesecond moments, the FI matrix can be further simplified as

$\begin{matrix}{{FIM} = {\sum\limits_{i = 1}^{n}{\frac{s^{2}}{I_{i}}B_{i}^{T}{B_{i}.}}}} & (10)\end{matrix}$

where B_(i) represents the i^(th) row of B∈

^(n×6). The weighted least square estimation can be efficientlyperformed by caching Hadamard products of each pair of the basis images.

Note that {tilde over (m)} and m denote second-moment outputs of themaximum likelihood estimator and the weighted least-square estimatorrespectively. The FI matrix assigns weights to each orientationalcomponent m_(k) inversely proportional to the expected measurementvariance of the DSFs used in SMOLM. Equation (2) may be minimized usingany existing method without limitation including, but not limited to,the f mincon function in MATLAB (Mathworks, R2019a). The eigenvectorcorresponding to the largest eigenvalue of the second-moment matrix maybe assigned as the initial orientation of the minimization of (2).

Quantifying the Precision of Measuring 3D Orientation and 3D Position

In one aspect, the Fisher information matrix quantifies the amount ofinformation contained within any DSF regarding the parameters to beestimated. The shape of the pixOL DSF contains information of both theemitter's axial location and its 3D orientation. Due to the discretesampling and finite size of the camera pixels, the off-grid distances[dx, dy, dh] related to the emitter's location [x, y, h] relative to theclosest grid point [x₀, y₀, h₀] also affect the shape of the DSF image.Therefore, the covariance between any pair of position and orientationparameters [x, y, h, θ, ϕ, Ω] is nonzero (FIG. 43 ). Thus, to properlyquantify the best-possible precision of measuring 3D position and 3Dorientation, while also considering correlations thereof, Fisherinformation matrices for joint estimation of 3D position and 3Dorientation can be calculated (FIG. 27A-D).

For FIGS. 27A and B, a 6×6 Fisher information matrix that includes 3location and 3 orientation parameters is calculated. By inverting theFisher information matrix, the CRB matrix K is obtained, whose diagonalelements quantify the estimation variance of 3D orientation [θ, ϕ, Ω]and 3D location [x, y, h]. To quantify the precision of estimating themean orientation [θ, ϕ], the angular precision σ_(δ) can be calculated,which is the half-angle of the uncertainty cone for estimating the meanorientation direction. It is a summary metric that combines theprecision σ_(θ) of measuring θ and the standard deviation σ_(ϕ) ofmeasuring ϕ, given by

$\begin{matrix}{{\sigma_{\delta} = {2{\arcsin\left( \sqrt{\frac{{\sin(\theta)}\sqrt{\det\left( K_{{4:5},{4:5}} \right)}}{4\pi}} \right)}}},} & ({S23})\end{matrix}$

where K_(4:5,4:5) is a 2×2 sub-matrix of K representing 3D orientation[θ, ϕ] precision formed by fourth and fifth rows and columns.Orientation space (θ, ϕ) is sampled uniformly using

ϕ=2πv ₁,  (S24)

θ=arccos(2u ₁−1),  (S25)

where v₁, u₁ are uniformly distributed on (0,1). The angular precisionσ_(δ) and wobbling angle precision σ_(Ω) can then be averaged over all θin FIGS. 27A and B. To quantify 3D location precision for isotropicemitters (FIGS. 27C and D), a 9×9 Fisher information matrix can bebuilt, which includes the 3 location parameters and 6 orientationalmoments. This formulation avoids the undefined mean orientationdirection (θ, ϕ) of these emitters. The correlation between the 3Dorientation and 3D location will degrade estimation precision ingeneral. The difference ratio Y between the precision calculated withorientation-position correlation and without is quantified using

$\begin{matrix}{{{Y(\sigma)} = \frac{\sigma_{{with}{correlation}} - \sigma_{{without}{correlation}}}{\sigma_{{without}{correlation}}}},} & ({S26})\end{matrix}$

where σ_(without correlation) is calculated using matrices K thatquantify solely 3D localization precision or 3D orientation measurementprecision. It is noticed that the performance of large DSFs (i.e., thosethat are greater than 3 times the size of the standard DSF) is lessinfluenced by orientation-position correlations (FIG. 44 ).Quantifying Estimation Bias when Measuring 3D Orientation and 3DPosition

To determine if correlations between measurements of 3D orientation and3D position influence estimation performance, images of emitters withvarious orientations and axial locations and a nominal focal plane setto z=−580 nm were simulated (FIG. 45-51 ). For each configuration, 200independent images are simulated using the forward model (Eqn. 1) andestimated using algorithms described herein. Overall, pixOL showsexcellent precision for measuring 3D orientation and 3D position.However, in some regions of the orientation domain, low estimationprecision and accuracy for both 3D orientation and 3D position isnoticed. Plotting the estimates in one of these regions, strongcorrelations between biases in 3D orientations and biases in 3Dlocations is noticed (FIGS. 53A and B). For example, simulated noiselessimages of an emitter at the ground truth position (x₀, y₀, h₀)=(0, 0,700) nm and orientation (θ₀, ϕ₀, Ω₀)=(38°, 162°, 0) are very similar tothose of a biased estimate of position and orientation (FIGS. 53C andD). However, differences between the noisy images are still discernible.Position-orientation correlations are suspected that create local minimain the likelihood space.

To make high-dimensional parameter estimation computationally feasible,the algorithm can iteratively update its approximate forward model basedon its current 3D position estimate (Algorithm 2 in FIG. 63B, step 10).This strategy leads to a non-continuous optimization surface andincreases the difficulty for the algorithm to jump out of local minima.It is possible that a neural network-based algorithm will be morecomputationally efficient and may be able to explore the 6Dposition-orientation space more robustly. Such an approach couldincrease estimation precision and remove bias (FIG. 53E).

Phase Masks Produced by DSF Optimization Using pixOL Method

To improve imaging sensitivity in SMOLM imaging, a phase mask at theback focal plane of a microscope may be used to modulate the detectedDSFs of each emitter. Applications that have a low signal-to-noise ratio(SNR) or densely labeled samples pose challenges to existing multi-spotDSFs. It is desirable to optimize a DSF for a specific sample ofinterest to achieve optimal imaging performance using DSF engineering.

An image of a fluorescent emitter, e.g., a molecule or nanoparticle,depends on its orientation. The image also contains information on howmuch a molecule rotates during a camera frame (called its wobbling). Asdescribed herein, individual fluorescent molecules are modeled asdipoles. It is assumed that a molecule rotates (wobbles) within asymmetric cone during one exposure time. As illustrated in FIG. 1 , θ,ϕmay be used to describe the center orientation of the cone and solidangle Ω [sr] may be used to describe the wobbling unit area on the unitsphere (Ω=0 means fixed dipole emitter and Ω=4π means a freely rotating,isotropic emitter). Although orientation information is contained withinthe light captured by a microscope, the traditional imaging system thatcreates the standard dipole spread function (DSF) cannot convey thisinformation to the image captured by a camera.

The image of an oriented emitter at the back focal plane of an objectivecan be decomposed into a linear combination of second-order orientationmoments (

u_(x) ²

,

u_(y) ²

,

u_(z) ²

,

μ_(xy)

,

μ_(xz)

, and

μ_(yz)

) with their corresponding basis images (see FIG. 4 ), where (μ_(x),μ_(y), μ_(z)) depicts a Cartesian coordinate projection of (θ,ϕ) and

⋅

represents average operator over camera frame. In various aspects, aphase mask may be designed based on the distribution of the basisfunction on the back focal plane.

In various aspects, an algorithm termed the pixOL method simultaneouslyoptimizes all pixels of a phase mask across two orthogonally polarizedimaging channels for applications with a low SNR. Unlike optimizationusing Zernike polynomial, pixOL can directly take advantage ofsuper-critical fluorescence arising from imaging SMs near a refractiveindex interface. Using the resulting phase mask provides forsimultaneous measurement of 3D orientation and 3D location of an emitterwith small statistical variance. The parameters in the pixOL algorithmmay be varied to optimize a phase mask to fit a range of microscopes andimaging applications.

In some aspects, a single-molecule microscope directly encodes anemitter's lateral location into the location of its DSF on the captured2D image. The axial location (h) and 3D orientation (θ, ϕ, Ω) are hiddenin the shape of the DSF, which may be manipulated using different phasemask designs as described herein. To achieve high estimation precisionfor 3D orientation and 3D location, the shape of the DSF is selected tovary quickly with respect to changes of the orientation and the axiallocation. This varying speed can be quantified using the Fisherinformation matrix (see Eqns. (9) and (10) above), which is only relatedto the microscope design and the imaging sample. Its inverse matrix,Cramér-Rao bound matrix (CRB), gives a lower bound on the estimationvariance for any unbiased estimator.

In various aspects, the CRB matrix K is used to estimate thesecond-moment vector m to optimize the phase mask for a microscope thatsplits the light into x-polarization and y-polarization channels (seeFIG. 2 ).

To calculate the best possible precision for estimating theorientational Moments m, the Cramér-Rao bound (CRB) matrix K can becalculated using Eqn. (S7) provided above.

In some embodiments, an optimal DSF that minimizes K for any possibleorientation [μ_(x), μ_(y), μ_(z), Ω] can be found to build a lossfunction

(Eqn. 11 below) to represent the sum of the precision of second momentsover a uniformly sampled orientation space M.

The mean orientation μ=[μ_(x), μ_(y), μ_(z)]^(T) can be sampled using

μ_(x)=2x ₁√{square root over (1−x ₁ ² −x ₂ ²)}  (S8a)

μ_(y)=2x ₂√{square root over (1−x ₁ ² −x ₂ ²)}  (S8b)

μ_(z)=1−2(x ₁ ² +x ₂ ²),  (S8e)

where x₁, x₂ are uniformly distributed within (−1, 1) and points forwhich x₁ ²+x₂ ²≥1 are rejected. The wobbling angle Ω can uniformlysampled within [0, 2π].

In some embodiments, using GradientTape for automatic differentiation inTensorflow, the gradient D of the loss function l with respect to thecurrent phase mask P can be calculated. In some embodiments, the maskfor dipole emitters located at the glass-water interface can beoptimized with 380 total signal photons detected and 2 backgroundphotons in each pixel. In some embodiments, the phase of each pixel canbe randomly initialized. The current phase mask can be updated using theAdam algorithm with a learning rate of 0.05 and a total of 300iterations, and the pixOL algorithm produces a phase mask exp(jP) shownin FIG. 26C and FIG. 35B.

In some aspects, the pixels P_(ij) of the phase mask P are allsimultaneously optimized by minimizing the loss function 1 given by:

$\begin{matrix}{l = {\min\limits_{P}{\sum_{m \in {\mathbb{C}}}\sqrt{K\left( {P,m} \right)}}}} & (11)\end{matrix}$

where

is a uniformly sampled orientation space. The algorithm outputs a phasemask (pixOL phase mask P_(opt)) that efficiently encodes the orientationmatrix m.

An overview of the disclosed pixOL method of designing a phase mask forSMOLM imaging is provided as FIG. 5 in one aspect. In this aspect, theloss function is minimized by step-wise updating a succession ofcandidate phase masks P_(t) based on the gradient d of the loss functionI with respect to the phase mask P

$\left( {d = \frac{\partial l}{\partial P}} \right)$

and reevaluating the resulting orientation matrix m and loss functionuntil a minimum loss function is achieved.

In some aspects, the phase mask resulting from the pixOL method may beimplemented using a spatial light modulator or other suitable device. Inone aspect, the phase mask may be defined directly from the optimizationresults as described above. In other aspects, the phase mask may bedefined using the conjugate of the optimal phase mask obtained using thepixOL method as described above.

SMOLM Imaging

In various aspects, any known SMOLM system may be used to perform SMOLMimaging, with the use of a DSF phase mask optimized using the disclosedmethods as described herein. Exemplary SMOLM systems are described indetail in U.S. Patent Application Publication 2018/0307132, the contentsof which are incorporated by reference herein in their entirety.

In one aspect, a home-built microscope with a 100× objective lens (NA1.40, Olympus, UPLSAPO100XOPSF) may be used to perform SMOLM imaging.For NR and MC540 imaging, a 561-nm laser (Coherent Sapphire) with a peakintensity of 1.31 kW/cm² and a dichroic beamsplitter (Semrock,Di03-R488/561) may be used. The emission may be filtered by a bandpassfilter (Semrock, FF01-523/610), and separated into x- and y-polarizedchannels by a polarization beam splitter (PBS, Meadowlark Optics,BB-100-VIS). The phase masks may be generated by a spatial lightmodulator (Meadowlark Optics, 256 XY Phase Series) onto which the backfocal plane of both polarization channels was projected. The modulatedSMOLM images may be captured with a typical 30 ms integration time usingan sCMOS camera (Hamamatsu ORCA-flash4.0 C11440-22CU).

Computing Systems and Devices

FIG. 6 depicts a simplified block diagram of a computing device forimplementing the methods described herein. As illustrated in FIG. 6 ,the computing device 300 may be configured to implement at least aportion of the tasks associated with the disclosed phase maskoptimization method using the SMLM system 310 including, but not limitedto: designing a phase mask using the disclosed phase mask optimizationmethod and operating the system 310 to obtain single-moleculelocalization microscopy (SMLM) images. The computer system 300 mayinclude a computing device 302. In one aspect, the computing device 302is part of a server system 304, which also includes a database server306. The computing device 302 is in communication with a database 308through the database server 306. The computing device 302 iscommunicably coupled to the SMLM system 310 and a user computing device330 through a network 60. The network 350 may be any network that allowslocal area or wide area communication between the devices. For example,the network 350 may allow communicative coupling to the Internet throughat least one of many interfaces including, but not limited to, at leastone of a network, such as the Internet, a local area network (LAN), awide area network (WAN), an integrated services digital network (ISDN),a dial-up-connection, a digital subscriber line (DSL), a cellular phoneconnection, and a cable modem. The user computing device 330 may be anydevice capable of accessing the Internet including, but not limited to,a desktop computer, a laptop computer, a personal digital assistant(PDA), a cellular phone, a smartphone, a tablet, a phablet, wearableelectronics, smartwatch, or other web-based connectable equipment ormobile devices.

In other aspects, the computing device 302 is configured to perform aplurality of tasks associated with designing a phase mask for obtainingSMLM images using the disclosed phase mask optimization method. FIG. 7depicts a component configuration 400 of computing device 402, whichincludes database 410 along with other related computing components. Insome aspects, computing device 402 is similar to computing device 302(shown in FIG. 6 ). A user 404 may access components of computing device402. In some aspects, database 410 is similar to database 308 (shown inFIG. 6 ).

In one aspect, database 410 includes SMLM imaging data 418 and algorithmdata 420. Non-limiting examples of suitable algorithm data 420 includeany values of parameters defining the optimization of the phase maskdesign and the analysis of SMLM imaging data, such as any of theparameters from the equations described herein.

Computing device 402 also includes a number of components that performspecific tasks. In the exemplary aspect, computing device 402 includesdata storage device 430, SMLM component 450, and communication component460. Data storage device 430 is configured to store data received orgenerated by computing device 402, such as any of the data stored indatabase 410 or any outputs of processes implemented by any component ofcomputing device 402. SMLM component 450 is configured to operate orproduce signals configured to design a phase mask for obtaining SMLMimages using the disclosed phase mask optimization method, to operate anSMLM device to obtain SMLM data, and to reconstruct the SMLM image basedon the SMLM data.

Communication component 460 is configured to enable communicationsbetween computing device 402 and other devices (e.g. user computingdevice 330 and IMRT system 310, shown in FIG. 6 ) over a network, suchas network 350 (shown in FIG. 6 ), or a plurality of network connectionsusing predefined network protocols such as TCP/IP (Transmission ControlProtocol/Internet Protocol).

FIG. 8 depicts a configuration of a remote or user computing device 502,such as user computing device 330 (shown in FIG. 6 ). Computing device502 may include a processor 505 for executing instructions. In someaspects, executable instructions may be stored in a memory area 510.Processor 505 may include one or more processing units (e.g., in amulti-core configuration). Memory area 510 may be any device allowinginformation such as executable instructions and/or other data to bestored and retrieved. Memory area 510 may include one or morecomputer-readable media.

Computing device 502 may also include at least one media outputcomponent 515 for presenting information to a user 501. Media outputcomponent 515 may be any component capable of conveying information touser 501. In some aspects, media output component 515 may include anoutput adapter, such as a video adapter and/or an audio adapter. Anoutput adapter may be operatively coupled to processor 505 andoperatively coupleable to an output device such as a display device(e.g., a liquid crystal display (LCD), organic light-emitting diode(OLED) display, cathode ray tube (CRT), or “electronic ink” display) oran audio output device (e.g., a speaker or headphones). In some aspects,media output component 515 may be configured to present an interactiveuser interface (e.g., a web browser or client application) to user 501.

In some aspects, computing device 502 may include an input device 520for receiving input from user 501. Input device 520 may include, forexample, a keyboard, a pointing device, a mouse, a stylus, atouch-sensitive panel (e.g., a touchpad or a touch screen), a camera, agyroscope, an accelerometer, a position detector, and/or an audio inputdevice. A single component such as a touch screen may function as bothan output device of media output component 515 and input device 520.

Computing device 502 may also include a communication interface 525,which may be communicatively coupleable to a remote device.Communication interface 525 may include, for example, a wired orwireless network adapter or a wireless data transceiver for use with amobile phone network (e.g., Global System for Mobile communications(GSM), 3G, 4G or Bluetooth) or other mobile data network (e.g.,Worldwide Interoperability for Microwave Access (WIMAX)).

Stored in memory area 510 are, for example, computer-readableinstructions for providing a user interface to user 501 via media outputcomponent 515 and, optionally, receiving and processing input from inputdevice 520. A user interface may include, among other possibilities, aweb browser and client application. Web browsers enable users 501 todisplay and interact with media and other information typically embeddedon a web page or a website from a web server. A client applicationallows users 501 to interact with a server application associated with,for example, a vendor or business.

FIG. 9 illustrates an example configuration of a server system 602.Server system 602 may include, but is not limited to, database server306 and computing device 302 (both shown in FIG. 6 ). In some aspects,server system 602 is similar to server system 304 (shown in FIG. 6 ).Server system 602 may include a processor 605 for executinginstructions. Instructions may be stored in a memory area 625, forexample. Processor 605 may include one or more processing units (e.g.,in a multi-core configuration).

Processor 605 may be operatively coupled to a communication interface615 such that server system 602 may be capable of communicating with aremote device such as user computing device 330 (shown in FIG. 6 ) oranother server system 602. For example, communication interface 615 mayreceive requests from user computing device 330 via a network 350 (shownin FIG. 6 ).

Processor 605 may also be operatively coupled to a storage device 625.Storage device 625 may be any computer-operated hardware suitable forstoring and/or retrieving data. In some aspects, storage device 625 maybe integrated into server system 602. For example, server system 602 mayinclude one or more hard disk drives as storage device 625. In otheraspects, storage device 625 may be external to server system 602 and maybe accessed by a plurality of server systems 602. For example, storagedevice 625 may include multiple storage units such as hard disks orsolid-state disks in a redundant array of inexpensive disks (RAID)configuration. Storage device 625 may include a storage area network(SAN) and/or a network attached storage (NAS) system.

In some aspects, processor 605 may be operatively coupled to storagedevice 625 via a storage interface 620. Storage interface 620 may be anycomponent capable of providing processor 605 with access to storagedevice 625. Storage interface 620 may include, for example, an AdvancedTechnology Attachment (ATA) adapter, a Serial ATA (SATA) adapter, aSmall Computer System Interface (SCSI) adapter, a RAID controller, a SANadapter, a network adapter, and/or any component providing processor 605with access to storage device 625.

Memory areas 510 (shown in FIG. 8 ) and 610 may include, but are notlimited to, random access memory (RAM) such as dynamic RAM (DRAM) orstatic RAM (SRAM), read-only memory (ROM), erasable programmableread-only memory (EPROM), electrically erasable programmable read-onlymemory (EEPROM), and non-volatile RAM (NVRAM). The above memory typesare examples only and are thus not limiting as to the types of memoryusable for storage of a computer program.

The computer systems and computer-implemented methods discussed hereinmay include additional, less, or alternate actions and/orfunctionalities, including those discussed elsewhere herein. Thecomputer systems may include or be implemented via computer-executableinstructions stored on non-transitory computer-readable media. Themethods may be implemented via one or more local or remote processors,transceivers, servers, and/or sensors (such as processors, transceivers,servers, and/or sensors mounted on vehicle or mobile devices, orassociated with smart infrastructure or remote servers), and/or viacomputer-executable instructions stored on non-transitorycomputer-readable media or medium.

In some aspects, a computing device is configured to implement machinelearning, such that the computing device “learns” to analyze, organize,and/or process data without being explicitly programmed. Machinelearning may be implemented through machine learning (ML) methods andalgorithms. In one aspect, a machine learning (ML) module is configuredto implement ML methods and algorithms. In some aspects, ML methods andalgorithms are applied to data inputs and generate machine learning (ML)outputs. Data inputs may further include: sensor data, image data, videodata, telematics data, authentication data, authorization data, securitydata, mobile device data, geolocation information, transaction data,personal identification data, financial data, usage data, weatherpattern data, “big data” sets, and/or user preference data. In someaspects, data inputs may include certain ML outputs.

In some aspects, at least one of a plurality of ML methods andalgorithms may be applied, which may include but are not limited to:linear or logistic regression, instance-based algorithms, regularizationalgorithms, decision trees, Bayesian networks, cluster analysis,association rule learning, artificial neural networks, deep learning,dimensionality reduction, and support vector machines. In variousaspects, the implemented ML methods and algorithms are directed towardat least one of a plurality of categorizations of machine learning, suchas supervised learning, unsupervised learning, and reinforcementlearning.

In one aspect, ML methods and algorithms are directed toward supervisedlearning, which involves identifying patterns in existing data to makepredictions about subsequently received data. Specifically, ML methodsand algorithms directed toward supervised learning are “trained” throughtraining data, which includes example inputs and associated exampleoutputs. Based on the training data, the ML methods and algorithms maygenerate a predictive function that maps outputs to inputs and utilizethe predictive function to generate ML outputs based on data inputs. Theexample inputs and example outputs of the training data may include anyof the data inputs or ML outputs described above.

In another aspect, ML methods and algorithms are directed towardunsupervised learning, which involves finding meaningful relationshipsin unorganized data. Unlike supervised learning, unsupervised learningdoes not involve user-initiated training based on example inputs withassociated outputs. Rather, in unsupervised learning, unlabeled data,which may be any combination of data inputs and/or ML outputs asdescribed above, is organized according to an algorithm-determinedrelationship.

In yet another aspect, ML methods and algorithms are directed towardreinforcement learning, which involves optimizing outputs based onfeedback from a reward signal. Specifically, ML methods and algorithmsdirected toward reinforcement learning may receive a user-defined rewardsignal definition, receive data input, utilize a decision-making modelto generate an ML output based on the data input, receive a rewardsignal based on the reward signal definition and the ML output, andalter the decision-making model so as to receive a stronger rewardsignal for subsequently generated ML outputs. The reward signaldefinition may be based on any of the data inputs or ML outputsdescribed above. In one aspect, an ML module implements reinforcementlearning in a user recommendation application. The ML module may utilizea decision-making model to generate a ranked list of options based onuser information received from the user and may further receiveselection data based on a user selection of one of the ranked options. Areward signal may be generated based on comparing the selection data tothe ranking of the selected option. The ML module may update thedecision-making model such that subsequently generated rankings moreaccurately predict a user selection.

As will be appreciated based upon the foregoing specification, theabove-described aspects of the disclosure may be implemented usingcomputer programming or engineering techniques including computersoftware, firmware, hardware, or any combination or subset thereof. Anysuch resulting program, having computer-readable code means, may beembodied or provided within one or more computer-readable media, therebymaking a computer program product, i.e., an article of manufacture,according to the discussed aspects of the disclosure. Thecomputer-readable media may be, for example, but is not limited to, afixed (hard) drive, diskette, optical disk, magnetic tape, semiconductormemory such as read-only memory (ROM), and/or any transmitting/receivingmedia, such as the Internet or other communication network or link. Thearticle of manufacture containing the computer code may be made and/orused by executing the code directly from one medium, by copying the codefrom one medium to another medium, or by transmitting the code over anetwork.

These computer programs (also known as programs, software, softwareapplications, “apps”, or code) include machine instructions for aprogrammable processor, and can be implemented in a high-levelprocedural and/or object-oriented programming language, and/or inassembly/machine language. As used herein, the terms “machine-readablemedium” “computer-readable medium” refers to any computer programproduct, apparatus and/or device (e.g., magnetic discs, optical disks,memory, Programmable Logic Devices (PLDs)) used to provide machineinstructions and/or data to a programmable processor, including amachine-readable medium that receives machine instructions as amachine-readable signal. The “machine-readable medium” and“computer-readable medium,” however, do not include transitory signals.The term “machine-readable signal” refers to any signal used to providemachine instructions and/or data to a programmable processor.

As used herein, a processor may include any programmable systemincluding systems using micro-controllers, reduced instruction setcircuits (RISC), application-specific integrated circuits (ASICs), logiccircuits, and any other circuit or processor capable of executing thefunctions described herein. The above examples are examples only, andare thus not intended to limit in any way the definition and/or meaningof the term “processor.”

As used herein, the terms “software” and “firmware” are interchangeableand include any computer program stored in memory for execution by aprocessor, including RAM memory, ROM memory, EPROM memory, EEPROMmemory, and non-volatile RAM (NVRAM) memory. The above memory types areexamples only and are thus not limiting as to the types of memory usablefor storage of a computer program.

In one aspect, a computer program is provided, and the program isembodied on a computer-readable medium. In one aspect, the system isexecuted on a single computer system, without requiring a connection toa server computer. In a further aspect, the system is being run in aWindows® environment (Windows is a registered trademark of MicrosoftCorporation, Redmond, Wash.). In yet another aspect, the system is runon a mainframe environment and a UNIX® server environment (UNIX is aregistered trademark of X/Open Company Limited located in Reading,Berkshire, United Kingdom). The application is flexible and designed torun in various different environments without compromising any majorfunctionality.

In some aspects, the system includes multiple components distributedamong a plurality of computing devices. One or more components may be inthe form of computer-executable instructions embodied in acomputer-readable medium. The systems and processes are not limited tothe specific aspects described herein. In addition, components of eachsystem and each process can be practiced independently and separate fromother components and processes described herein. Each component andprocess can also be used in combination with other assembly packages andprocesses. The present aspects may enhance the functionality andfunctioning of computers and/or computer systems.

Definitions and methods described herein are provided to better definethe present disclosure and to guide those of ordinary skill in the artin the practice of the present disclosure. Unless otherwise noted, termsare to be understood according to conventional usage by those ofordinary skill in the relevant art.

In some embodiments, numbers expressing quantities of ingredients,properties such as molecular weight, reaction conditions, and so forth,used to describe and claim certain embodiments of the present disclosureare to be understood as being modified in some instances by the term“about.” In some embodiments, the term “about” is used to indicate thata value includes the standard deviation of the mean for the device ormethod being employed to determine the value. In some embodiments, thenumerical parameters set forth in the written description and attachedclaims are approximations that can vary depending upon the desiredproperties sought to be obtained by a particular embodiment. In someembodiments, the numerical parameters should be construed in light ofthe number of reported significant digits and by applying ordinaryrounding techniques. Notwithstanding that the numerical ranges andparameters setting forth the broad scope of some embodiments of thepresent disclosure are approximations, the numerical values set forth inthe specific examples are reported as precisely as practicable. Thenumerical values presented in some embodiments of the present disclosuremay contain certain errors necessarily resulting from the standarddeviation found in their respective testing measurements. The recitationof ranges of values herein is merely intended to serve as a shorthandmethod of referring individually to each separate value falling withinthe range. Unless otherwise indicated herein, each individual value isincorporated into the specification as if it were individually recitedherein. The recitation of discrete values is understood to includeranges between each value.

In some embodiments, the terms “a” and “an” and “the” and similarreferences used in the context of describing a particular embodiment(especially in the context of certain of the following claims) can beconstrued to cover both the singular and the plural, unless specificallynoted otherwise. In some embodiments, the term “or” as used herein,including the claims, is used to mean “and/or” unless explicitlyindicated to refer to alternatives only or the alternatives are mutuallyexclusive.

The terms “comprise,” “have” and “include” are open-ended linking verbs.Any forms or tenses of one or more of these verbs, such as “comprises,”“comprising,” “has,” “having,” “includes” and “including,” are alsoopen-ended. For example, any method that “comprises,” “has” or“includes” one or more steps is not limited to possessing only those oneor more steps and can also cover other unlisted steps. Similarly, anycomposition or device that “comprises,” “has” or “includes” one or morefeatures is not limited to possessing only those one or more featuresand can cover other unlisted features.

All methods described herein can be performed in any suitable orderunless otherwise indicated herein or otherwise clearly contradicted bycontext. The use of any and all examples, or exemplary language (e.g.“such as”) provided with respect to certain embodiments herein isintended merely to better illuminate the present disclosure and does notpose a limitation on the scope of the present disclosure otherwiseclaimed. No language in the specification should be construed asindicating any non-claimed element essential to the practice of thepresent disclosure.

Groupings of alternative elements or embodiments of the presentdisclosure disclosed herein are not to be construed as limitations. Eachgroup member can be referred to and claimed individually or in anycombination with other members of the group or other elements foundherein. One or more members of a group can be included in, or deletedfrom, a group for reasons of convenience or patentability. When any suchinclusion or deletion occurs, the specification is herein deemed tocontain the group as modified thus fulfilling the written description ofall Markush groups used in the appended claims.

Any publications, patents, patent applications, and other referencescited in this application are incorporated herein by reference in theirentirety for all purposes to the same extent as if each individualpublication, patent, patent application, or other reference wasspecifically and individually indicated to be incorporated by referencein its entirety for all purposes. Citation of a reference herein shallnot be construed as an admission that such is prior art to the presentdisclosure.

Having described the present disclosure in detail, it will be apparentthat modifications, variations, and equivalent embodiments are possiblewithout departing the scope of the present disclosure defined in theappended claims. Furthermore, it should be appreciated that all examplesin the present disclosure are provided as non-limiting examples.

EXAMPLES

The following examples illustrate various aspects of the disclosure.

Example 1: Pixol: Pixel-Wise Dipole Spread Function Engineering forMeasuring the 3D Orientation and 3D Location of Dipole-Like Emitters

Here an algorithm is proposed, termed pixOL, to simultaneously optimizeall pixels of a phase mask across two orthogonally polarized imagingchannels for applications with a low SNR (FIG. 11 ). Unlike optimizationusing Zemike polynomials, pixOL can directly take advantage ofsuper-critical fluorescence arising from imaging SMs near a refractiveindex interface. Using the resulting phase mask (FIG. 11 inset) enablessimultaneous measurement of 3D orientation and 3D location of an emitterwith small statistical variance. One can easily modify the parameters inpixOL to optimize a phase mask to fit their own microscope andapplication. Using the pixOL DSF, the orientation of Nile red (NR)molecules transiently attached to two supported lipid bilayers (SLBs):DPPC (di(16:0) PC) and DPPC with cholesterol is measured. Theorientations of NR measured by the pixOL DSF reveal the chemicalcompositions of the SLBs.

The SM is modeled as a dipole-like emitter (FIGS. 25A and 25B). Eachemitter's 3D location is represented by (y, y, h). Polar angle θ andazimuthal angle ϕ describes the mean orientation of the emitter during acamera frame, and the solid angle Ω [sr] describes the wobble of amolecule during the camera acquisition (Ω=0 represents a fixed dipoleemitter and Ω=2 pi means a freely rotating, isotropic emitter). For eachset of orientation angles (θ, ϕ, Ω), there is a unique time-averagedorientational second moment MεR^(6×1). The pixOL algorithm optimizes thephase mask P with N×N pixels by simultaneously updating all pixelstowards the gradient direction that minimizes the Cramér-Rao (CRB)matrix R_(M)εR^(6×6) as

P _(opt)=argmin_(P∈R) _(N×N) Σ√(det(R _(M)(θ,ϕ,Ω,P))),

where det(•) donates the matrix determinant, and the summation is overorientation space. Thus, pixOL converges upon the phase mask thatoptimizes the precision of measuring M.

With the pixOL phase mask Popt (FIG. 11 inset) inserted at the backfocal plane of a microscope, emitters at different 3D orientations(FIGS. 25C and D, emitters 1-6) and at two axial locations (FIGS. 25Cand D, h=O nm and h=300 nm) are easily resolvable from one another inthe image plane. Notably, these images are similar in size to thestandard diffraction-limited DSF. Thus, the pixOL DSF is suitable forlow SNR applications and densely labeled samples. For fixed emitters(Ω=0) at a glass-water interface, the average root-mean-square angularerror σ_(k) is calculated as a combined standard deviation of measuringθ and ϕ and the average standard deviation σ_(Ω) of Ω using the CRB(FIG. 13 two left graphs). Compared to other orientation-sensing DSFs,namely CHIDO, the polarized DSF defocused at 200 nm below the coverslip,and the tri-spot DSF, the pixOL DSF shows the best 3D orientationestimation precision. The lateral (x,y) localization precision σ_(r) andthe axial localization precision σ_(h) for an isotropic emitter (FIG. 13, two right graphs) were also calculated. The pixOL DSF has the bestaxial localization precision and good lateral precision following CHIDO.

Using the pixOL DSF, the orientations of Nile red (NR) emitterstransiently attached to two SLBs: DPPC (di(16:0) PC) and DPPC with 40%cholesterol were measured (FIGS. 25A and B). Within SLBs, theorientation of NR is greatly influenced by its local environment. Addedcholesterol (chol) condenses lipid acyl chains and stabilizes lipidmembranes. A sparsity-promoting maximum-likelihood estimator was used toestimate the brightness, 2D position (x,y), and 3D orientation (θ, ϕ, Ω)of individual NR within each frame. The orientations of NRs in DPPC withchol are tilted further away from the coverslip plane than NRs in DPPCwithout chol (FIG. 25B). The orientations in DPPC with chol have anarrower distribution indicating a more stabilized lipid membraneenvironment (FIG. 25C).

Here, an algorithm (pixOL) that optimizes a phase mask pixel-by-pixel toefficiently encode the 3D orientation and 3D location of an emitter intothe shape of the DSF is demonstrated. The resulting pixOL DSF, optimizedfor a microscope with two polarized detection channels, shows superiormeasurement precision for both 3D orientation and 3D position comparedto existing methods. In addition, the pixOL DSF is similar in size tothe diffraction-limited DSF, enabling it to be used with samplesoptimized for standard localization microscopy with minimal changes. Theorientation measurements of NR confirm that pixOL can visualize thechemical compositions of lipid membranes.

Example 2: Pixel-Wise Optimization for 3D Orientation and 3DLocalization Estimation

A fluorescent emitter is modeled as a dipole-like emitter withorientation characterized by the mean orientation direction (θ, ϕ) andthe wobbling solid angle Ω (FIG. 26B). The image produced by themicroscope is linearly proportional to a molecule's orientationalsecond-moment vector m=[

μ_(x) ²

,

μ_(y) ²

,

μ_(z) ²

,

μ_(x)μ_(y)

,

μ_(x)μ_(z)

,

μ_(y)μ_(z)

]∈R⁶ as I=s [B_(xx), B_(yy), B_(zz), B_(xy), B_(xz), B_(yz)]m+b∈

^(N×1), where

^(N×1) is the captured intensity on a camera with N pixels, s is thenumber of signal photons detected from the emitter, and b is thebackground in each pixel. The matrices B_(j)∈

^(N×6), correspond to the imaging system's response to eachorientational second moment and can be calculated with vectorialdiffraction theory; these basis images comprise the 6 modes of anyfluorescence microscope when imaging dipole-like emitters. The anglebrackets

⋅

represent a temporal average over one camera frame

A single-molecule microscope directly encodes an emitter's laterallocation into the location of its DSF on the captured 2D image. Theaxial location h and 3D orientation (θ, ϕ, Ω) are hidden in the shape ofthe DSF. To achieve high estimation precision for 3D orientation and 3Dlocation, the shape of the DSF should vary quickly with respect tochanges in the orientation and the axial location of the emitter. Thisvarying speed can be quantified using Fisher information matrix, whichis only related to the microscope design and the imaging sample. Itsinverse matrix, Cramér-Rao bound matrix (CRB), gives a lower bound onthe estimation variance for any unbiased estimator.

The CRB matrix K is leveraged for estimating the second moment vector nto optimize the phase mask for a microscope that splits the light intox-polarization and y-polarization channel (FIG. 26A). We consideremitters that are located at the focal plane and water environment areconsidered. The refractive index interface creates intensityconcentrated super-critical angles at the back focal plane. To bestleverage this feature, all the pixels Pij of the phase mask aresimultaneously optimized by minimizing the loss function l (Eqn. (11)above).

To avoid poor lateral estimation precision, The algorithm is forced tocreate DSFs smaller than 1.8 μm×1.8 μm by ignoring photons outside thisrange. The algorithm outputs a phase mask (pixOL phase mask) thatefficiently encodes the orientational six moments m.

With the pixOL phase mask (FIG. 26C) at the back focal plane, themicroscope generates DSFs with various shapes and different intensitydistribution across x/y channels for SMs with different orientations andaxial locations (FIG. 26D). Interestingly, a 90° rotation of DSFs forSMs above the coverslip and below the coverslip were noticed, indicatinghigh estimation precision for axial location.

Using CRB of three orientation parameters or location parameters, thepixOL DSF is compared to other engineered DSFs designed for 3Dorientation namely the polarized DSF defocused at 200 nm below thecoverslip, tri-spot DSF, and CHIDO DSF which is designed for 3Dorientation and 3D localization. The mean square angular error a iscalculated as a combined standard deviation of measuring θ, ϕ and thestandard deviation σ_(Ω) of wobbling angle. For emitters at the focalplane (FIGS. 27A and B) or out of the focal plane, pixOL shows the bestprecision for measuring the three orientation parameters. The standarddeviation a of measuring the lateral location and the standard deviationσ_(h) of height for isotropic emitters across an axial range of 800 nmwere also quantified (FIGS. 26C and D). The pixOL DSF has the best axialestimation precision and good lateral precision. Therefore, it isexpected that pixOL DSF will be a good technique for simultaneouslyorientation and localization estimation.

In any experiment, optical aberrations will perturb the designed DSF anddecrease estimation performance. When using a liquid-crystal spatiallight modulator to create the piXOL DSF, the conjugate of the pixOLphase mask (pixOL_(cjg)) preserves the features from the ideal DSFbetter than pixOL phase mask itself. Therefore, the conjugate phase maskwas used in the experimental demonstrations. A custom algorithm based onnegative log likelihood loss function is used for joint orientation andlocation estimation. To calibrate the algorithm, phase retrieval wasused to model the experimental DSF calibrated using fluorescent beads.

Scanning these beads across an axial range of 1400 nm enablesverification of the localization and orientation measurement precisionexperimentally. The estimated focal plane locations z resolve the 50 nmstage movements very well (FIG. 27E) with an axial precision σ_(z) of2.56 nm averaged over all steps (FIG. 27E). Orientation measurements ofthe bead confirm that its emission pattern is consistent with that of anisotropic emitter (FIG. 27E, average Ω=1.76π, average σ_(Ω)=0.08).Measurement for other beads confirms similar results. These observationsconfirm that the pixOL DSP's estimates of position and orientation areaccurate and precise for bright emitters (average of XX photonsdetected).

Example 3: 3D Orientation and 3D Location Estimations Reveal LipidComposition

Optical Instrumentation and Alignment

The pixOL DSF is implemented using a home-built epifluorescencemicroscope. Briefly, a polarization resolved 4f imaging system,consisting of relay lenses and a polarizing beamsplitter, is appended toa fluorescence microscope to project two polarized images onto separateregions of a camera (FIG. 30 ). A spatial light modulator (SLM,Meadowlark Optics, 256 XY Phase Series) is placed at the conjugate backfocal plane (BFP) of the imaging system and loaded with the pixOL phasemask to modulate the x- and y-polarized fluorescence simultaneously(FIG. 30 inset(i)(ii)).

To properly align the SLM, we load a binary phase mask consisting ofconcentric rings of increasing radius (10-pixel interval) centered onthe SLM. A dense layer of fluorescent beads (100-nm diameter red 580/605FluoSpheres, Invitrogen F8801) on a coverslip is used to generatefluorescence for calibration. By using the flipping mirror in FIG. 30 ,a camera (camera 2 in FIG. 30 ) placed at a conjugate back focal planeis able to observe the pupil of the objective lens and the SLM ringpattern simultaneously in the y-polarization channel. The axialalignment of the SLM is adjusted until the pupil and rings are sharpsimultaneously. The SLM is shifted laterally until the SLM's ring maskis centered with respect to the microscope pupil (FIG. 54 ). Thealignment of the SLM within the x-polarization channel can verified byadjusting the position of lens 4 and camera 2 in FIG. 30 . In ourexperience, lateral deviations of the SLM up to ˜0.1 mm are difficult todetect by observing the pixOL DSF itself.

Imaging System Calibration

We calibrate the imaging model in Section 2 to the DSF of our imagingsystem by using fluorescent beads (100-nm diameter red 580/605FluoSpheres, Invitrogen F8801). Images are captured by scanning theobjective's nominal focal plane from z=−790 nm to z=610 nm with a stepsize of 50 nm (FIGS. 55E, 56F, and 27E). At each plane, 11 images aretaken.

A phase-retrieval algorithm is used to retrieve the experimental phasemask, calibrated pixOL ^(˜)P. To accurately characterize the opticalaberrations of our microscope, the phase masks of the two polarizedchannels are estimated independently (FIGS. 55B and C, FIGS. 56C and D).The simulated DSFs using the calibrated phase masks ^(˜)P match theexperimental DSFs very well (FIGS. 55F and 56G).

Phase retrieval for both the pixOL phase mask and its conjugate mask(pixOL*) was performed. The retrieval algorithm assumes that thepolarized field from a dipole emitter is collected by an ideal objectivelens and that the polarizing beam splitter splits x and y-polarizedlight with a perfect contrast ratio. Thus, the optical aberrations areassumed to be strictly phase-only. It is noticed that the pixOL phasemask's DSF has a large aberration when the objective is focused belowthe coverslip (z>0, FIG. 55E). While aberrations also exist in thepixOL* experimental DSF at the same locations, its mismatch relative tothe ideal pixOL DSF is more modest and is similar to sphericalaberration (FIG. 56F).

High frequency patterns in the calibrated phase masks near the edge ofthe aperture are also noticed (FIGS. 55B and C, FIGS. 56C and D). Totest if these patterns meaningfully impact the shape of the DSF,smoothed versions of the calibrated phase masks are reconstructed byprojecting it into a Zernike basis using the first 231 Zernikepolynomials (FIGS. 57A and B). The negative-likelihood (NLL, Eqn. S15)is calculated between the DSF predicted by each mask and theexperimental images of beads used for phase retrieval (FIG. 55D and FIG.56E). The NLL (NLL_(calibrated)) for the calibrated phase masks isalways smaller than the NLL (NLL_(smoothed)) for the smoothed phasemasks (FIGS. 57C and D). These data indicate that the calibrated masksare more consistent with the experimentally observed DSFs than thesmoothed masks.

Preparing 3D Spherical Supported Lipid Bilayers

Supported lipid bilayers (SLBs) were formed by fusing vesicles on silicabeads. A lipid mixture containing 0.286 mg/mL DPPC (di(16:0)phosphatidylcholine, Avanti Polar Lipids 850355) and 0.1 mg/mLcholesterol (Sigma Aldrich C8667) within Tris buffer (100 mM NaCl, 3 mMCa2+, 10 mM Tris, pH 7.4) is first incubated in a water bath at atemperature of 65° C. Simultaneously, 1 mg/mL silica beads (2.0 μmdiameter, Bangs Laboratories SS04002) are also incubated in the waterbath for ˜10 minutes. After two solutions reach to thermal equilibrium,30 μL of the lipid mixture and 90 μL of the silica beads are mixedtogether and stay in the water bath for another 30 minutes. During the30 minutes, the mixture is vortexed every 5 minutes. The mixture is thenmoved outside the water bath together with around 200 mL of warm waterto allow it to cool down to room temperature for about 1 hour. Duringthis cooling process, the mixture is vortexed every 5 minutes to allowthe lipid bilayer to coat the bead uniformly.

After the mixture cooled down to room temperature, the excess vesiclesare removed using six successive 5-min centrifuge spins at 500 RPM. Thesupernatant is discarded and replaced with imaging buffer (100 mM NaCl,10 mM Tris, pH 7.4) after each spin.

Imaging Spherical Lipid Bilayers Using Nile Red

Nile red (32 nM) is added to the imaging buffer to probe the morphologyand composition of the spherical lipid bilayers. Circularly polarizedillumination (1533 W/cm2 or 30 mW at 561 nm at the sample) is used toexcite NR. Fluorescence is collected using an objective lens (OLYMPUSUPLSAPO100XOPSF, NA 1.4) and passes through a dichroic mirror(Di01-R488/561, Semrock) and bandpass filter (FF01-523/610, Semrock).The difference between the basis images of the pixOL DSF forfluorescence at the two edges of the filter (641 nm and 583 nm) is small(peak difference of 8.3%, FIG. 58 ). The chromatic aberration introducedby the nonzero fluorescence bandwidth is negligible.

20000 frames are captured with an exposure time of 110 ms. To compensatefor axial drift of the microscope during long-term imaging, theobjective is refocused every 2000 frames. Nominal focal plane (NFP) ofz=−350 nm for the DPPC+chol bead and an NFP of z=−500 nm for theDPPC-only bead is used. The lateral drift of the microscope is correctedafter imaging. The lateral centers of each sphere are aligned using SMposition estimates averaged in groups of 2000 frames. With theseexperimental conditions, single molecule blinking are observed withproperties shown in FIG. 60 .

To quantify the localization precision for NRs, we plot the NR_(lateral)positions r relative to the sphere's center within each h slice (FIGS.61D and 62D). We compare the experimental distribution to a theoreticaldistribution p_(theo) calculated by convolving the lateral positionsp_(sphere) representing the spherical surface with the expected laterallocalization distribution p_(pixOL)* of pixOL* based on the CRB whennormal focal plane equals to −580 nm, i.e.

p _(theo)(r)=p _(sphere)

p _(pixOL)*  (S27)

where

${{p_{sphere}(r)} = \frac{R}{\left( \sqrt{R^{2} - r^{2}} \right)}},{{p_{{pixOL}^{*}}(r)} = {\frac{1}{{\sigma_{L}\left( \sqrt{R^{2} - r^{2}} \right)}\sqrt{2\pi}} \times {\exp\left\lbrack {{- \frac{1}{2}}\left( \frac{r}{\sigma_{L}\left( \sqrt{R^{2} - r^{2}} \right)} \right)^{2}} \right\rbrack}}}$

-   -   where R is the radius of the sphere, σ_(L)(√{square root over        (R²−r²)}) is the lateral precision of pixOL* for an emitter on        the sphere's surface with h=R−√{square root over (R²−r²)}, and        represents convolution operation.

Results

To validate the pixOL technique for simultaneous 3D orientation and 3Dlocation estimation, supported lipid bilayers (SLBs) are adhered to 2 μmsilica beads (FIG. 28A). In here, two kinds of SLBs are used, namelyDPPC with 40% cholesterol and DPPC. Nile reds (NRs) are used totransiently attached to the SLBs and emit fluorescent light (FIG. 28A).A quarter waveplate (QWP) is added into the imaging pathway to generatecircular polarized light so that both in-plane and out-of-plane emitterscan be excited.

For beads coated with DPPC and cholesterol, the estimated orientationchanges from out-of-plane orientations (small θ) at the bottom of thesphere to be in-plane orientations (θ closing to 90°) at the waist planeof the sphere (FIGS. 28B and C). The estimated ϕ changes radially alongthe circle at each z slices (FIG. 28E). For estimations below the waistplane of the sphere (h=1000 nm), the ϕ estimation matches to theopposite direction of the colormap; when SMs are close to the waistplane, the orientation overlapped for every pair of ϕ and ϕ-180° as theyare identical when θ=90°; for SMs above the waist plane, the ϕ becomesmatching to the colormap. The 3D orientation (θ, ϕ) indicates that theNRs are nearly perpendicular bounds to the spherical surface of beads,which is consistent to the orientation measurement for NRs that bound toSLBs coating on a 2D coverslip. The raw images also show largesimilarity to the simulated DSFs for SMs with perpendicular orientationto the spherical surface at three axial locations (FIG. 28G). Toquantify the experimental 3D localization estimation precision, we plotthe histograms of the distance r between the estimated location to thevertical diameter of the sphere for SMs at different z slices (FIG.28F). The theoretical distribution using the convolution between thedistribution of the spherical model and the precision of pixOLcjg wasalso calculated Based on the full width half maximum (FWHM), the lateralprecision for DPPC and DPPC with cholesterol data are pretty close tothe theoretical precision (FIG. 28H): the experimental estimations givea mean FWHM of 134 nm and the theoretical distribution gives a mean of82 nm across a 1200 nm axial range.

Without cholesterol, the SLBs on the sphere are less condensed. Theorientation estimation for a bead coated with DPPC validated this as theorientation measurement (θ, ϕ) shows a more random distribution than theDPPC with cholesterol sample (FIG. 28D). Comparing the wobbling angle Ωand angle θ⊥, which is the angle between the estimated orientation andthe vector perpendicular to the spherical surface (FIG. 28I), DPPC withcholesterol sample gives a relative small θ⊥ and Ω, indicatingperpendicular orientation to the spherical surface and more fixedorientation, while DPPC sample gives a broader θ⊥ distribution and largewobbling Ω. Therefore, measuring the orientations of Nile reds withpixOL can easily tell the absence of cholesterol as the orientationsenses the local environment difference.

What is claimed is:
 1. A computer-implemented method for producing anoptical mask for an SMOLM imaging system, the method comprising: a.providing, to a computing device, a baseline optical mask comprising aplurality of mask pixels distributed at a plurality of mask pixelpositions within a mask plane, each mask pixel comprising at least oneoptical modulation element configured to modulate at least one opticalparameter of a photon produced by an emitter propagating therethrough;b. providing, to the computing device, a plurality of emitter imagesindicative of dipole spread functions captured using the SMOLM imagingsystem provided with the baseline optical mask, each image comprising aplurality of image pixels indicative of a dipole spread function, eachimage pixel comprising a pixel position and a pixel intensity indicativeof a number of photons detected at the pixel position, wherein eachemitter image is obtained for a reference emitter positioned at areference lateral position and at one sample orientation within anorientation space; c. determining, using the computing device, a lossfunction comprising a matrix quantifying variances in precision ofemitter orientations estimated from the dipole spread functions from theplurality of images; d. iteratively modifying, using the computingdevice, at least one optical parameter of at least one mask pixel tominimize the loss function to produce the optical mask.
 2. The method ofclaim 1, wherein the at least one optical parameter modulated by eachmask pixel comprises a phase, a polarization, a birefringence, and anycombination thereof.
 3. The method of claim 2, wherein the opticalparameter modulated by each mask pixel is the phase.
 4. The method ofclaim 1, wherein the plurality of emitter images are obtained by: a.imaging an emitter at a plurality of orientations within the orientationspace using the SMOLM imaging system provided with a baseline opticalmask or a modified optical mask; or b. simulating each emitter imageusing a computational model of a SMOLM imaging system provided with abaseline optical mask or a modified optical mask.
 5. The method of claim4 wherein each emitter image is simulated using a dipole-dipole model.6. The method of claim 1, wherein the matrix quantifying variances inprecision of emitter orientations estimated from the dipole spreadfunctions from the plurality of images comprises a Cramér-Rao boundmatrix K, wherein the Cramér-Rao bound matrix quantifies a lower boundon a variance of estimated emitter orientations.
 7. The method of claim6, wherein the loss function l is given by:$l = {\min\limits_{P}{\sum\limits_{m \in {\mathbb{C}}}\sqrt{K\left( {P,m} \right)}}}$wherein m is a second-moment vector m of a dipole spread functionobtained using the optical mask P, and

is a uniformly sampled emitter orientation space.
 8. The method of claim1, further including producing additional optical masks for images of anemitter positioned at different axial positions within the SMOLM imagingsystem, for images of an emitter comprising background photons, forimages of emitters positioned out of the focal plane of the SMOLMimaging system.
 9. The method in claim 1 wherein the loss function isquantified and minimized by a divergence statistical model.
 10. Themethod of claim 1 wherein the each emitter image of the plurality ofimages is indicative of at least two dipole spread functionscorresponding to at least two emitters within each image.
 11. Acomputer-implemented method for producing a phase mask for an SMOLMimaging system, the method comprising: e. providing, to a computingdevice, a baseline phase mask comprising a plurality of mask pixelsdistributed at a plurality of mask pixel positions within a mask plane,each mask pixel comprising a phase modulation element configured tomodulate the phase of a photon produced by an emitter propagatingtherethrough; f. providing, to the computing device, a plurality ofemitter images indicative of dipole spread functions captured using theSMOLM imaging system provided with the baseline optical mask, each imagecomprising a plurality of image pixels indicative of a dipole spreadfunction, each image pixel comprising a pixel position and a pixelintensity indicative of a number of photons detected at the pixelposition, wherein each emitter image is obtained for a reference emitterpositioned at a reference lateral position and at one sample orientationwithin an orientation space; g. determining, using the computing device,a loss function comprising a matrix quantifying variances in precisionof emitter orientations estimated from the dipole spread functions fromthe plurality of images; h. iteratively modifying, using the computingdevice, at least one optical parameter of at least one mask pixel tominimize the loss function to produce the phase mask.
 12. The method ofclaim 11, wherein the plurality of emitter images are obtained by: a.imaging an emitter at a plurality of orientations within the orientationspace using the SMOLM imaging system provided with a baseline opticalmask or a modified optical mask; or b. simulating each emitter imageusing a computational model of a SMOLM imaging system provided with abaseline optical mask or a modified optical mask.
 13. The method ofclaim 12 wherein each emitter image is simulated using a dipole-dipolemodel.
 14. The method of claim 11, wherein the matrix quantifyingvariances in precision of emitter orientations estimated from the dipolespread functions from the plurality of images comprises a Cramér-Raobound matrix K, wherein the Cramér-Rao bound matrix quantifies a lowerbound on a variance of estimated emitter orientations.
 15. The method ofclaim 14, wherein the loss function l is given by:$l = {\min\limits_{P}{\sum\limits_{m \in {\mathbb{C}}}\sqrt{K\left( {P,m} \right)}}}$wherein m is a second-moment vector m of a dipole spread functionobtained using the optical mask P, and

is a uniformly sampled emitter orientation space.
 16. The method ofclaim 11, further including producing additional optical masks forimages of an emitter positioned at different axial positions within theSMOLM imaging system, for images of an emitter comprising backgroundphotons, for images of emitters positioned out of the focal plane of theSMOLM imaging system.
 17. The method in claim 11 wherein the lossfunction is quantified and minimized by a divergence statistical model.18. The method of claim 11 wherein the each emitter image of theplurality of images is indicative of at least two dipole spreadfunctions corresponding to at least two emitters within each image.